[Usage]: 'InternVLChatConfig' object has no attribute 'num_attention_heads'

Your current environment
The output of `python collect_env.py`

PyTorch version: 2.3.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: CentOS Linux release 7.9.2009 (Core) (x86_64)
GCC version: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)
Clang version: Could not collect
CMake version: version 3.29.5
Libc version: glibc-2.17
Python version: 3.11.0 (main, Mar  1 2023, 18:26:19) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-3.10.0-1160.92.1.el7.x86_64-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA L20
GPU 1: NVIDIA L20
GPU 2: NVIDIA L20
GPU 3: NVIDIA L20
GPU 4: NVIDIA L20
GPU 5: NVIDIA L20
GPU 6: NVIDIA L20
GPU 7: NVIDIA L20
Nvidia driver version: 535.154.05
cuDNN version: Probably one of the following:
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn.so.8.9.7
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.9.7
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.9.7
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.9.7
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.9.7
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.9.7
/usr/local/cuda-12.1/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.9.7
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                128
On-line CPU(s) list:   0-127
Thread(s) per core:    2
Core(s) per socket:    32
座：                 2
NUMA 节点：         2
厂商 ID：           GenuineIntel
CPU 系列：          6
型号：              143
型号名称：        Intel(R) Xeon(R) Gold 6462C
步进：              8
CPU MHz：             3900.018
CPU max MHz:           3900.0000
CPU min MHz:           800.0000
BogoMIPS：            6600.00
虚拟化：           VT-x
L1d 缓存：          48K
L1i 缓存：          32K
L2 缓存：           2048K
L3 缓存：           61440K
NUMA 节点0 CPU：    0-31,64-95
NUMA 节点1 CPU：    32-63,96-127
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb cat_l3 cat_l2 cdp_l3 invpcid_single cdp_l2 ssbd mba rsb_ctxsw ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq cldemote movdiri movdir64b md_clear pconfig spec_ctrl intel_stibp flush_l1d arch_capabilities
Versions of relevant libraries:
[pip3] numpy==1.26.3
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] torch==2.3.0+cu121
[pip3] torchaudio==2.3.0+cu121
[pip3] torchvision==0.18.0+cu121
[pip3] transformers==4.41.2
[pip3] transformers-stream-generator==0.0.5
[pip3] triton==2.3.0
[conda] numpy                     1.26.3                   pypi_0    pypi
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] torch                     2.3.0+cu121              pypi_0    pypi
[conda] torchaudio                2.3.0+cu121              pypi_0    pypi
[conda] torchvision               0.18.0+cu121             pypi_0    pypi
[conda] transformers              4.41.2                   pypi_0    pypi
[conda] transformers-stream-generator 0.0.5                    pypi_0    pypi
[conda] triton                    2.3.0                    pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.5.0.post1
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    GPU1    GPU2    GPU3    GPU4    GPU5    GPU6    GPU7    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      PIX     SYS     SYS     SYS     SYS     SYS     SYS     0-31,64-95      0               N/A
GPU1    PIX      X      SYS     SYS     SYS     SYS     SYS     SYS     0-31,64-95      0               N/A
GPU2    SYS     SYS      X      PIX     SYS     SYS     SYS     SYS     0-31,64-95      0               N/A
GPU3    SYS     SYS     PIX      X      SYS     SYS     SYS     SYS     0-31,64-95      0               N/A
GPU4    SYS     SYS     SYS     SYS      X      PIX     SYS     SYS     32-63,96-127    1               N/A
GPU5    SYS     SYS     SYS     SYS     PIX      X      SYS     SYS     32-63,96-127    1               N/A
GPU6    SYS     SYS     SYS     SYS     SYS     SYS      X      PIX     32-63,96-127    1               N/A
GPU7    SYS     SYS     SYS     SYS     SYS     SYS     PIX      X      32-63,96-127    1               N/A
Legend:
X    = Self
SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX  = Connection traversing at most a single PCIe bridge
NV#  = Connection traversing a bonded set of # NVLinks
How would you like to use vllm
I used the swift deploy command to deploy internvl-chat-v1_5, and an error was reported. The command is as follows:
CUDA_VISIBLE_DEVICES=6,7 swift deploy --model_type internvl-chat-v1_5 --model_id_or_path /data/pretrained_models/AI-ModelScope/InternVL-Chat-V1-5  --port 23333  --infer_backend vllm
The error message is:

But I changed --infer_backend from vllm to pt and it can be deployed normally.
The following is my installation environment:
ms-swift=2.1.0
torch=2.3.0+cu121
vllm=0.5.0.post1
vllm-flash-attn=2.5.9
flash-attn=2.5.9.post1
Is it because vllm does not support internvl-chat-v1_5 when the above error message appears?

@wangdong1992 does your model have a valid config.json in the folder you are using? vLLM needs to read the config.json to know how to construct the model. Here is an example from a HF checkpoint https://huggingface.co/OpenGVLab/InternVL-Chat-V1-5/blob/main/config.json

@wangdong1992 does your model have a valid config.json in the folder you are using? vLLM needs to read the config.json to know how to construct the model. Here is an example from a HF checkpoint https://huggingface.co/OpenGVLab/InternVL-Chat-V1-5/blob/main/config.json
I confirmed that there is a valid config.json in my model. Does vllm support internvl 1.5 now?

Does vllm support InternVL-Chat-V1-5 now?

I also reported the same error as you

'InternVLChatConfig' object has no attribute 'num_attention_heads' same error

'InternVLChatConfig' object has no attribute 'num_attention_heads' same error

I had this same error with InternVL2

same error

you could check your vllm version, vllm>=0.5.4

same error

same error

See if it's fixed in vLLM 0.6.0. Which InternVL model are you using?


Error when consuming iso returned from a function to use as parameter for another function

Use case: I have function A which does some work and returns an Array[U8] iso. I then want to take that and consume it as an argument for function B which is also expecting an Array[U8] iso. This results in a compiler error.
Code:
Playpen link: http://pony-playpen.lietar.net/?gist=fa2f2f5f8647cd146aa32c3605dc6ad0
actor Main
  new create(env:Env) =>
    let d = f()

    let d' = g(consume iso d)

  fun f(): Array[U8] iso =>
    let d = recover Array[U8] end
    d.push(4)
    d.push(4)
    d.push(4)
    d.push(4)
    d

  fun g(data: Array[U8] iso) =>
    None

Error:
0.14.0 [release]
compiled with: llvm 3.9.1 -- cc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Error:
main.pony:5:16: argument not a subtype of parameter
    let d' = g(consume iso d)
               ^
    Info:
    main.pony:7:12: Array[U8 val] iso! is not a subtype of Array[U8 val] iso: iso! is not a subcap of iso
      fun f(): Array[U8] iso =>
               ^

Additional info:
Changing the iso in g(consume iso d) and g(data: Array[U8] iso) to trn results in a similar error:
0.14.0 [release]
compiled with: llvm 3.9.1 -- cc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Error:
main.pony:5:16: argument not a subtype of parameter
    let d' = g(consume trn d)
               ^
    Info:
    main.pony:7:12: Array[U8 val] trn! is not a subtype of Array[U8 val] trn: trn! is not a subcap of trn
      fun f(): Array[U8] iso =>
               ^

Changing the iso in g(consume iso d) and g(data: Array[U8] iso) to val or ref compiles successfully.
Changing let d' = g(consume iso d) to let d' = consume iso d compiles successfully.
Changing the code to remove the function f() like:
actor Main
  new create(env:Env) =>
    let d = recover Array[U8] end
    d.push(4)
    d.push(4)
    d.push(4)
    d.push(4)
    d

    let d' = g(consume iso d)

  fun g(data: Array[U8] iso) =>
    None

also compiles successfully.

@dipinhora - It's not necessarily intuitive, but you need to change the signature of f to specify Array[U8] iso^ - specifying Array[U8] iso is not enough.

Thanks for the pointer.
Is it possible to improve the error message to at least suggest that the programmer that adding the ^?

Sure, should be possible. I'll give it a shot.

@dipinhora I've opened #2004 to improve the error message:
/home/jemc/1/code/gitx/ponyc/test.jemc/test.pony:5:16: argument not a subtype of parameter
    let d' = g(consume iso d)
               ^
    Info:
    /home/jemc/1/code/gitx/ponyc/test.jemc/test.pony:7:12: Array[U8 val] iso! is not a subtype of Array[U8 val] iso: iso! is not a subcap of iso
      fun f(): Array[U8] iso =>
               ^
    /home/jemc/1/code/gitx/ponyc/test.jemc/test.pony:7:22: this would be possible if the subcap were more ephemeral
      fun f(): Array[U8] iso =>
                         ^

@jemc Thanks. Much appreciated.


Ensure proper replacement

Any LIKE %%-statement with another Where-statement is currently not properly checked. This PR resolves that.
Example:
query = s.db.Table("accounts").
    Where("accounts.firstName LIKE ?", "%John%").


Testing, please ignore. (zgyjsp)

Please ignore.. Please DO NOT APPROVE THIS! (zgyjsp)

Hi @helumanxa,
Thank you for contributing this pull request!
Please sign the Kaltura CLA so we can review and merge your contribution.
Learn more at http://bit.ly/KalturaContrib


Audit when an instance is destroyed because it failed the healthcheck initialization timeout

As the title says.
From slack:
@alena: you do have this:
```2016-04-29 20:50:56,048 INFO  [:] [] [] [] [ecutorService-2] [.h.s.i.HealthcheckCleanupMonitorImpl] Scheduling remove for instance id [186]

i just missed it bc it was buried

Alena [3:35 PM]
yes

[3:36]
I should add “removing because initializing timeout is hit”

Craig Jellick [3:36 PM]
that makes sense

[3:36]
darren mentioned that you audit a lot of the things u do with healthchecks. looks like this one is not audited. maybe add this one to audit?

[3:36]
not necessary, just a thought if ur goal was to have a full audit of all healthcheck related events

Alena [3:37 PM]
it is not audited

[3:37]
could you file it for me?

Closing due to inactivity, please reopen if you think this is still relevant to current releases & important.


(internal) release_patch: build v5.10.1164

Test plan: automated release PR, CI will perform additional checks

:green_circle: Internal release is ready for promotion!
:warning: Do not close/merge the pull request or delete the associated branch if you intend to promote it.

:green_circle: Release has been promoted, see tag: https://github.com/sourcegraph/deploy-sourcegraph-k8s/tree/v5.10.1164.
:no_entry: Do not under any circumstance delete the branch holding the tagged commit (i.e. release-v5.10.1164).
:arrow_right: You can safely close the PR and delete its a associated branch.


SEN12MS baseline

repo
wandb
pretrained models are downloaded and placed on s3 (s3://sen12ms/pretrained_sup) for faster download along with other pre-trained models.

use the pretrained models
run various prediction and explore different metrics for baselines

Once this is done, please convert this issue to the discussion for documentation purpose.

started new evaluation on the baseline pre-training models (supervised)
first one (single label)

single_label/single_DenseNet121_s1s2
single_label/single_DenseNet121_s2
single_label/single_ResNet50_RGB
the link is showing blank. can I check if this is the right link? thanks

https://wandb.ai/cal-capstone/SEN12MS/table?workspace=user-taeil
For both baseline and CLS training evaluation (from my perspective)

Must tracked metrics (loss, mircro f1, macro f1, top-1 accuracy, top-1 precision, top-1 recall, class-wise accuracy)
To be checked with Colorado that whether these needs to be reported ( top - {2...}acc, f{2..}, overall acc, average acc)

Thanks


simulating joint backlash

Hi mujoco gurus,
I'm an engineer and I'm trying to use MuJoCo for sim2real transfer in Go1 quadruped.
I'm looking for some help with backlash simulation.
Here is a model that we created to simulate the backlash following this page as well as joint equality:
<xml>
    <default>
        <joint damping="1.0" armature="0.01" frictionloss="0.2" />

        <default class="backlash">
            <joint damping="0.0" armature="0.0" frictionloss="0.0"
                   limited="true" range="-0.04 0.04"/>
        </default>

        <equality solref=".1 1" solimp="0.1 0.99 0.04 0.5 3.0" />
    </default>

    <worldbody>
        <joint name="FR_calf_joint" pos="0 0 0" axis="0 1 0" limited="true" range="-2.69653 -0.916298" />
        <joint name="FR_calf_joint_b" pos="0 0 0" axis="0 1 0" class="backlash" />
    </worldbody>

    <equality>
        <joint joint1="FR_calf_joint_b"/>
    </equality>

</xml>

However, we are not able to create the backlash effect in simulation (blue curve).  Green is physical readings from a step response (dashed line is the command, and each tick in x-axis is 5ms).
See screenshot:

We confirmed with the manufacturer that the sensor readings are from the output gears, but there could be mechanical interference from other parts (e.g. a lever after the output gear) causing this move-stop-move effect (which looks like pre-gear backlash).  The plotted simulation curves are from sensor readings of the joint.  In such a case, what parameters can we tweak to simulate this "backlash" effect?
Is it possible to have the sensor read out the pre-gear readings, so it's similar to the real backlash?
We've also tried plotting the sensor readings as well as the qpos of the two joints (original and shadow), they do not seem to make sense to us.  Sensor reading != qpos + qpos_b.  Could someone explain what's going on?  How is sensor reading computed, is it based on qpos and qpos_b?
2023-07-19 15:14:54.288 | INFO     | foxy.mujoco_agent:read:180 - --------------------
2023-07-19 15:14:54.288 | INFO     | foxy.mujoco_agent:read:181 - sensor = -1.7749549857460385
2023-07-19 15:14:54.288 | INFO     | foxy.mujoco_agent:read:182 - qpos   = -1.769121879011308
2023-07-19 15:14:54.288 | INFO     | foxy.mujoco_agent:read:183 - qpos_b = -0.04833542130998575
2023-07-19 15:14:54.307 | INFO     | foxy.mujoco_agent:read:180 - --------------------
2023-07-19 15:14:54.307 | INFO     | foxy.mujoco_agent:read:181 - sensor = -1.769121879011308
2023-07-19 15:14:54.307 | INFO     | foxy.mujoco_agent:read:182 - qpos   = -1.76386839600179
2023-07-19 15:14:54.307 | INFO     | foxy.mujoco_agent:read:183 - qpos_b = -0.04874816118409169
2023-07-19 15:14:54.307 | INFO     | foxy.mujoco_agent:read:180 - --------------------
2023-07-19 15:14:54.307 | INFO     | foxy.mujoco_agent:read:181 - sensor = -1.76386839600179
2023-07-19 15:14:54.307 | INFO     | foxy.mujoco_agent:read:182 - qpos   = -1.7592229302175098
2023-07-19 15:14:54.307 | INFO     | foxy.mujoco_agent:read:183 - qpos_b = -0.04923959357205287
2023-07-19 15:14:54.308 | INFO     | foxy.mujoco_agent:read:180 - --------------------
2023-07-19 15:14:54.308 | INFO     | foxy.mujoco_agent:read:181 - sensor = -1.7592229302175098
2023-07-19 15:14:54.308 | INFO     | foxy.mujoco_agent:read:182 - qpos   = -1.7552189310423365
2023-07-19 15:14:54.308 | INFO     | foxy.mujoco_agent:read:183 - qpos_b = -0.049796946315089664
2023-07-19 15:14:54.308 | INFO     | foxy.mujoco_agent:read:180 - --------------------
2023-07-19 15:14:54.308 | INFO     | foxy.mujoco_agent:read:181 - sensor = -1.7552189310423365
2023-07-19 15:14:54.308 | INFO     | foxy.mujoco_agent:read:182 - qpos   = -1.751879114829859
2023-07-19 15:14:54.308 | INFO     | foxy.mujoco_agent:read:183 - qpos_b = -0.05040694388060367
2023-07-19 15:14:54.327 | INFO     | foxy.mujoco_agent:read:180 - --------------------
2023-07-19 15:14:54.327 | INFO     | foxy.mujoco_agent:read:181 - sensor = -1.751879114829859
2023-07-19 15:14:54.327 | INFO     | foxy.mujoco_agent:read:182 - qpos   = -1.7492145623362705
2023-07-19 15:14:54.327 | INFO     | foxy.mujoco_agent:read:183 - qpos_b = -0.05105544032121813
Thanks,
Le
Ps, the backlash happens during the step response of the quadruped standing up, not when crouching down.  This is very counter intuitive, as one would think that backlash should happen in the reverse direction of the gears' tension, so should happen when it is crouching down.  This strange behavior is observed in both calf and thigh joints.  We're following up with the manufacture for this.

We stopped using joint velocity as input, relying only on frame stacked joint position.  This, plus all the domain randomizations reduces the policy's sim2real gap.


Fix Issue 17322 - Add Magikcraft to organizations using D

CC @jwulf - could you please check this? (you can preview the changes with DAutoTest).
Also do you by chance have a good resource about your usage of D that we could link?
Maybe have a blog post, web page, talk, ...

Ping @jwulf - still in the our (D) universe?

Hey, this looks good. I gave D a shout-out at NDC Oslo in my talk. Will drop a link when the talk video goes up.

@jwulf thanks for looking over this. FYI: it's now live & Magikcraft is listed at: http://dlang.org/orgs-using-d.html

Hey, this looks good. I gave D a shout-out at NDC Oslo in my talk. Will drop a link when the talk video goes up.

Thanks. Looking forward :)


Add error constants

This PR exposes additional constants from the pycares.errno module.  I added tests to verify that we are exporting all of the constants as well.

[Current coverage][cc-pull] is 94.35%

Merging [#1][cc-pull] into [master][cc-base-branch] will increase coverage by 1.42%

@@             master         #1   diff @@
==========================================
  Files             1          1
  Lines            99        124    +25
  Methods           0          0
  Messages          0          0
  Branches          9          9
==========================================
+ Hits             92        117    +25
  Misses            4          4
  Partials          3          3

Powered by Codecov. Last updated by [6c3071d...0bfc856][cc-compare]
[cc-base-branch]: https://codecov.io/gh/gmr/tdns/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/gmr/tdns/compare/6c3071dc390a1f201d7c0a12711fd42af5224240...0bfc8560424b67093c97cf959c4d00acfc39e37c
[cc-pull]: https://codecov.io/gh/gmr/tdns/pull/1?src=pr

Thanks!

Tagged


ScreenManager broken if JSON data is invalid

Bug Report
On a specific Ford SYNC IVI we have seen that the system is returning invalid JSON data to media apps if they change templates. Once they change back to the MEDIA template it looks like sending text is not possible anymore. The app shows an error in the logs where the library catches an NPE caused by this line of code https://github.com/smartdevicelink/sdl_java_suite/blob/4.12.0/base/src/main/java/com/smartdevicelink/managers/ManagerUtility.java#L77 The error message is

java.lang.NullPointerException: Attempt to invoke virtual method 'int com.smartdevicelink.proxy.rpc.enums.TextFieldName.ordinal()' on a null object reference

The root cause is based on invalid JSON data in the .name parameter of a TextField. That'll cause ordinal() to fail.
Suggested fix
While we attempt to fix the issue in the head unit there is need for a library workaround. The easiest is to skip TextField objects where getName() returns null. However, that skipped field could be mainField4 which would be skipped, therefore the method would return 3 but it's an unusable result. In this unfortunate situation of a null name, we would suggest to return 4 and write a warning to the logs.
Reproduction Steps

Use a test media app with 4.12 (or even 4.11)
Connect the test app to the concerning IVI
Show some text using screen manager
SetDisplayLayout TILES_WITH_GRAPHIC
Show some other text using screen manager

Expected Behavior
The new text should be shown
Observed Behavior

Nothing's happening
The error message is shown in the app logs

OS & Version Information

Android Version: Any (not relevant)
SDL Android Version: 4.11 or 4.12
Testing Against: Test app, SYNC3 lite

Test Case, Sample Code, and / or Example App
Available on request

Closing as duplicate of #1465 as well as already fixed with two PRs #1474 & #1498.
The behavior desired on this issue as requested by the reporter would result in a poorer UX for scenarios outside the specific case defined. The 3 lines vs 4 lines does not create an unusable result as the reporter states; it creates a less than perfect result that still works. However, if the library took null as an assumption that 4 lines of text are available even if we have only received up to mainField3 we have no confirmation that 4 lines of texts are available. If only 3 lines were available on the IVI system this would result in an unusable UX as the library assumes 4 lines when only 3 are available.
So while it would be a best case solution for a specific scenario where the RPC spec is not followed, it would create extremely poor and unusable UX for other scenarios.

@joeygrover thank you for sharing the links. I focused on previous releases and didn't take a closer look to develop.
To my understanding mainField4 isn't missing on our system but regarding your response I must say it's wrong. The computed result is unusable because you can't know if mainFields are missing therefore the method can't promise knowledge of maximum number of text fields. null is not an assumption of four lines but affects confidence to the result; with null and mainField3 you don't have 100% confirmation that only 3 lines are available. Hence, on systems where 4 lines would be available your implementation behaves inconsistently.
It is up to you to decide which side you want to support. Defending the decision you made is one thing. Affecting application behavior is another.

The logic in the library is written with the assumption that the IVI system properly supplies its capabilities and that it follows the RPC spec. If that system doesn’t adhere to those requirements, it will have to bear the consequences. Null is not an acceptable value and should be ignored in the case where other fields have been processed successfully.
The scenario you stated is statistically improbable. The IVI system was able to successfully create and send a TextField item with name mainfield3 but was not able to do so for mainfiled4 while still being able to form a proper JSON object. This would only really be possible if the IVI system improperly described their capabilities which is not something the library should protect for.
The situation you are actually seeing is where the IVI system sends a value for TextFieldName that is not an accepted/valid value of the RPC spec enum. So the JSON object is properly formed with an incorrect value. This situation demonstrates why an incorrect value should not be the reason to consider an RPC as unusable. Take SDL Core for example, it simply strips out values it doesn’t know about to process what it can, that same logic can be applied here.
If any SDLC member has an IVI system that is demonstrating the behavior you stated it can be discussed further. However, creating a workaround solution for a hypothetical problem is not the best course of action, nor is creating one that, as previously stated, would create an unusable UX as compared to a less than perfect one.

I think you don't really understand my point that the computed data has no 100% confidence because it knows some data is wrong/corrupted. Specifically to text fields and capabilities the use case is that the head unit returns which fields are visible with the currently selected template. Any other field can be send but will not be presented. That's what the library logic is trying to prevent. Sending more content has no affect to the result. A head unit which would show more content than described in capabilities is more questionable than a bug producing null in the list.
I'm surprised about your arguments not comparing both sides and weigh objectively. It's obvious that you want to defend the decisions you have made in the past but should accept feedback if these decisions are weak in certain scenarios.

Feedback is always welcomed and encouraged. It can be easy to review code ad hoc and not realize how it works within the larger system. This is especially true when trying to handle errors. The logic of how to handle the situation with this issue was thought through properly to give the best average case experience.
I think there is a misunderstanding of what this method is used for and how changing it would impact what's seen on screen. Let's take the example of a developer setting 4 lines of text through the screen manager as follows:

Line 1: Hi
Line 2: my name
Line 3: is
Line 4: SDL

Now let's take a look at how that would appear in the different scenarios  we've discussed.
1. Best case scenario: Text fields are properly described!
TextFields:[mainfield1, mainfield2, mainfield3, mainfield4]

mainfield1: Hi
mainfield2: my name
mainfield3: is
mainfield4: SDL

Looks exactly as expected.
2. Next let's look at what would happen with null or unsupported values while the HMI actually supported all 4 main fields. as the logic stands today.
TextFields:[mainfield1, mainfield2, notificationText, null]

mainfield1: Hi - my name
mainfield2: is - SDL
mainfield3:
mainfield4:

As you can see, the developer still gets all the information on screen in this scenario. Since we were confident that at least main fields 1 and 2 were available we used those to display all of the information. Obviously this isn't the best case scenario for this HMI, but with either improperly describing HMI capabilities, not following the RPC spec, or having JSON data being corrupted this is an acceptable experience.
3. When we look at what happens if the HMI truly only supports 2 lines but also includes null or unsupported values.
TextFields:[mainfield1, mainfield2, notificationText, null]

mainfield1: Hi - my name
mainfield2: is - SDL

This is the same outcome as if the array only included mainfield1 & mainfield2 which is correct.
4. Finally let's take look at what would happen in scenario 3 but changing the logic to throw out the textfield array since a null or incorrect value was present.
TextFields:[mainfield1, mainfield2, notificationText, null]

mainfield1: Hi
mainfield2: my name

As you can see not all of the information is passed to the HMI and is very likely to create an unusable experience for the user. While the example text given is very trivial, things like turn-by-turn instructions or possibly weather updates would fill out all 4 lines and without combining those lines the user would miss potentially vital information.
In conclusion, the logic does not prevent information from being sent to the HMI, it is actually the exact opposite. The logic in this particularly method ensures that the developer is able to get all main field lines displayed regardless of quantity supported (or assumed supported). One final case to be aware of is that if the capabilities as a whole are null, which is a much more likely scenario in the case the JSON payload is corrupted, the library defaults to 4 lines. If you have any evidence of a JSON object being corrupted in the specific way that it is still able to be formed into a JSON object but only values within that object are affected to the point they can't be recognized as subsequent JSON objects, I would be very eager to find out what's causing that and how we can mitigate that issue.


💥FIX : Missing Mouse animation added

Title of the Pull Request

[x] Have you renamed the PR Title in a meaningful way?

Related Issue

Closes: #
Description
Missing mouse animation added on Testimonial.
How Has This Been Tested? ⚙
Go to the Testimonial page and check animation .
Screenshots 📷

Type of change
What sort of changes have you made:

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)

Checklist

[x] My code follows the code style of this project.
[x] I have followed the contribution guidelines
[x] I have performed a self-review of my own code.
[x] I have ensured my changes don't generate any new warnings or errors.
[x] I have updated the documentation (if necessary).
[x] I have resolved all merge conflicts.

@vishanurag please review and merge the PR.


Multi version generalization

@nkubala @cftorres @aaron-prindle
#2
(Ideally to be merged before #1 so that pip differ is working accurately before being expanded to more package locations)

lgtm


Merge main into devel

Merging main into devel to keep devel up-to-date

You can try but I think now (with your Write role), you should be able to rebase devel on top of main and force-push devel in the repo. If that does not work I can do it myself too. That's just to avoid the merge commit and keep the history linear but if we don't care that much we can just merge this.

I guess it's up to you.  The rebase will be annoying, replaying each of the commits in devel onto main.  At some point devel will merge back into main.
I tend to prefer merges, but again, up to you :)

I just rebased devel on top of main and force-pushed the branch. It's currently passing CI now, hope it will pass.
I figured it will be easier when we decide to lift the code-freeze because we will just be able to fast-forward the main branch with devel and delete devel.

So the problem with rebasing is that you lose all the merge history in devel.  Not sure if it matters.

Yeah not sure, as long as you don't lose commit history I'd say it's ok


couldn't able to find ServiceaccountIAMBinding in provider-gcp-cloudplatform provider

What problem are you facing?

How could Official GCP Provider help solve your problem?

You can use https://marketplace.upbound.io/providers/upbound/provider-gcp-cloudplatform/v1.5.0/resources/cloudplatform.gcp.upbound.io/ServiceAccountIAMMember/v1beta1


New package: ComplexPlots v0.1.0

Registering package: ComplexPlots
Repository: https://github.com/complexvariables/ComplexPlots.jl
Created by: @tobydriscoll
Version: v0.1.0
Commit: 4f31a04c1c8f8e8fb02ea081818e855769fc837e
Reviewed by: @tobydriscoll
Reference: https://github.com/complexvariables/ComplexPlots.jl/commit/4f31a04c1c8f8e8fb02ea081818e855769fc837e#commitcomment-125854183
Description: Plotting in the complex plane for Julia.

This looks very interesting, but the docs-badges in the README currently return a 404 error. Can you make sure the documentation is working, before registering? Or else, at least add a usage example to the README.

Apologies, I had to figure out how to make Makie work with the Documenter builder. The dev docs are up now. I think/hope that once Tagbot gets to run, then the stable version will go up, but I will double check that.

Tobin Driscoll
Unidel Chaired Professor of Mathematical Sciences
Director, Master of Science Program in Data Science
University of Delaware,  Newark, DE 19716
Zoom: udel.zoom.us/my/tobydriscoll
Site: tobydriscoll.net

On Aug 30, 2023, at 12:44 AM, Michael Goerz @.***> wrote:
This looks very interesting, but the docs-badges in the README currently return a 404 error. Can you make sure the documentation is working, before registering? Or else, at least add a usage example to the README.
—
Reply to this email directly, view it on GitHub https://github.com/JuliaRegistries/General/pull/90443#issuecomment-1698484390, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA3JNLQBWZL5WCUIUJPTEUDXX3ALRANCNFSM6AAAAAA4DL3NNY.
You are receiving this because you were mentioned.

[noblock] Very nice, thanks!
You’ll want to edit your comment to include [noblock] so it doesn’t block merging


Cone Gate Creation

This is Mr. Hommowun
We printed off a "gate" that will connect to a servo on the bot for this Saturday. Are we able to program that and have it ready by then?

Can you provide more details as to what the cone gate entails? If It is a simple servo design I can definitely have that programmed before Friday. Please provide pictures if possible


Fix typo in README for dev build

I think this reflects what you asked for, and simplifies the installation instructions @mrakitin.


Match Dashboard frame to existing Docs style

Why
In order to preserve a seamless experience from dashboard to docs — and to better frame the dashboard — the content frame should have a consistent top border.
Acceptance Criteria

[ ] The top border of the main content frame should be the same in the dashboard as in the docs
[ ] The lead header of the content frame should be a consistent h2

Before

After

Addressed with the following PRs:
cg-deck https://github.com/18F/cg-deck/pull/342
cg-style https://github.com/18F/cg-style/pull/91


can't dl videos from france3-regions.francetvinfo.fr

I can't dl videos from france3-regions.francetvinfo.fr
Here are the logs :
youtube-dl --verbose "france3-regions.francetvinfo.fr/limousin/emissions/jt-1920-limousin.html"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'france3-regions.francetvinfo.fr/limousin/emissions/jt-1920-limousin.html']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.08.01
[debug] Python version 2.7.10 - Linux-4.2.0-42-generic-x86_64-with-Ubuntu-15.10-wily
[debug] exe versions: avconv 2.7.6-0ubuntu<IP_ADDRESS>, avprobe 2.7.6-0ubuntu<IP_ADDRESS>, ffmpeg 2.7.6-0ubuntu<IP_ADDRESS>, ffprobe 2.7.6-0ubuntu<IP_ADDRESS>, rtmpdump 2.4
[debug] Proxy map: {}
WARNING: The url doesn't specify the protocol, trying with http
[francetvinfo.fr] jt-1920-limousin: Downloading webpage
ERROR: Unable to extract video id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 682, in extract_info
ie_result = ie.extract(url)
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 347, in extract
return self._real_extract(url)
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/francetv.py", line 229, in _real_extract
webpage, 'video id').split('@')
File "/usr/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 650, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)
RegexNotFoundError: Unable to extract video id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

There is no video at http://france3-regions.francetvinfo.fr/limousin/emissions/jt-1920-limousin.html.


CTHUB-51:ZEVA - Update Supplementary Report Edit Form

Set default values to supplementary data if present otherwise set default value to existing assessment data.

Created new PR directed to new release branch so don't need this anymore


[DM-23547] Upgrade to cert-manager 0.13.1

Import new CRDs from the upstream Git repository and update the
cert-manager-crd app to deploy them.  Add some operational docs for
why this is a separate app and how it is updated.
Update all issuers to use the new API group and the new syntax for
specifying the solver.
Update all annotations to use the new annotation label for specifying
the issuer.
Remove the Argo CD ignoreDifferences configuration to see if this
problem is fixed with the latest cert-manager.
Remove the disable-validation annotation for the cert-manager
namespace since that upstream documentation page no longer exists
and upstream no longer appears to recommend this change.

Awesome work. I guess there isn't an immediate risk of failure if this change goes sideways because we already have certs in place, right? So we can watch the logs and generally see that cert-manager is healthy to know it's working.

Ah, I see the notes about deployment on the Jira ticket.


medusa-auth-plugin not working with 1.8.8 extender

Describe the bug
The callback from auth plugin is never handled and the admin callback function is never called. For example the following URL will just result in a failureUrl of the plugin being called. This worked fine in extender 1.8.2.
http://localhost:9000/admin/auth/auth0/cb?code=abc&state=def
To Reproduce
Steps to reproduce the behavior:

Configure auth0 in medusa-auth-plugin
Use extender 1.8.8
Validate successfully using auth0
failureUrl will be called instead of successUrl

Expected behavior
admin callback function is called after successful authentication
Package version:

1.8.8

I don't see what the extender could do to generate that behaviour. Another company is using auth0 with the extender 1.8.8 and everything is working as expected. Are you sure that calling the cb manually with random values will work?

I wasn't calling it with random values, but obfuscated them just to post publicly.
I will investigate more, but the only difference in the app breaking/not breaking is the extender version.

Sorry i thought that to test it you were manually calling the cb route. Can you share your config please

@Arsenalist do you have any more info?

Sorry haven’t had time. I’ll close it and re-open when I do get a chance. For now I have downgraded to 1.8.2 and it work fine.
I think the sequence of middleware execution changes had an impact but I haven’t looked deeply into it.

Got it, but there is no order of middleware that has changed. If it is required auth, the auth middleware is applied and then the handlers are called


PE-1429/add-pr-review-tasklist

This PR was generated using turbolift.

CircleCI is not running on this repo because it is stale. It does not seem worth the effort of re-connecting CircleCI to add this check at this time. Feel free to re-open/merge this PR if you think otherwise.


[BEAM-10650] OrderedListState API

This PR adds the API and and in-memory implementation for the timestamp-ordered list state. The API is currently marked experimental and is still subject to change.

 If you see any more things that should be changed, please comment and I will open a new PR to address those comments.


Revert "Enhance error message from kubectl when 'kind' is empty"

Reverts GoogleCloudPlatform/kubernetes#8404 - merge window was closed.

Forgot we were in a merge close window.


Delete CLI does not handle case sensitivity for DataFlow & DataJob

Describe the bug
The delete CLI does a exact match on "dataflow" and "datajob" which results in confusion when users input the entity_type as "dataJob" or "dataFlow"
To Reproduce
Steps to reproduce the behavior:
Use datahub delete  --entity_type dataFlow --platform airflow
Expected behavior
Should allow the command
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):
N/A
Additional context
Slack Thread

fixed by #4492


WIP - Pydantic v2 support

@collerek  Is this your up-to-date wip branch? are you open to getting some help with this?

Checklist:

[ ] Fix types
[ ] Fix linting
[ ] Make docs versionable to display docs for v1 and v2 in github pages
[ ] Update docs to reflect breaking changes (i.e. Meta is now OrmarConfig) and removed features in v2 (i.e. choices - use Enum field instead)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (fc9287e) 100.00% compared to head (444d418) 100.00%.
Report is 91 commits behind head on master.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

@@            Coverage Diff             @@
##            master     #1238    +/-   ##
==========================================
  Coverage   100.00%   100.00%
==========================================
  Files          200       200
  Lines        16572     15575   -997
==========================================
- Hits         16572     15575   -997

Files
Coverage Δ

ormar/decorators/__init__.py
100.00% <ø> (ø)

ormar/decorators/signals.py
100.00% <100.00%> (ø)

ormar/fields/__init__.py
100.00% <100.00%> (ø)

ormar/fields/base.py
100.00% <100.00%> (ø)

ormar/fields/constraints.py
100.00% <100.00%> (ø)

ormar/fields/foreign_key.py
100.00% <100.00%> (ø)

ormar/fields/many_to_many.py
100.00% <100.00%> (ø)

ormar/fields/model_fields.py
100.00% <100.00%> (ø)

ormar/fields/parsers.py
100.00% <100.00%> (ø)

ormar/fields/sqlalchemy_encrypted.py
100.00% <100.00%> (ø)

... and 171 more

I'll work on  Switch from on_startup and shutdown e to lifecycle (tests, docs). I will probably send PR after the weekend.


My app is ins German Language and I can’t change to English

I just downloaded the app, my iPad Pro default language is Brazilian Portuguese.
The app is in German, and I need to change to English, but I don’t have a clue on how to change, as I don’t understand German.
Is this a bug? Or someone can help me changing the language?

I'm not sure how this happened but you can go to Settings App > Code App and select another language.

@bummoblizard that worked, thanks!
An hypothesis I had was that as my main language (portuguese) the default language is being applied, and maybe the default is german.

@bummoblizard  that worked, thanks!
An hypothesis I had was that as my main language (portuguese) the default language is being applied, and maybe the default is german.

I’m not sure how this happened but you can go to Settings App > Code App
and select a new language there.
On Fri, 7 Oct 2022 at 02:51, Guima Ferreira @.***>
wrote:

I just downloaded the app, my iPad Pro default language is Brazilian
Portuguese.
The app is in German, and I need to change to English, but I don’t have a
clue on how to change, as I don’t understand German.
Is this a bug? Or someone can help me changing the language?
—
Reply to this email directly, view it on GitHub
https://github.com/thebaselab/codeapp/issues/603, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJE6T22EMJHHHQDRHZZE6SLWB4NRZANCNFSM6AAAAAAQ65BHDI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


fix #540 Error stack

Can you do this for all exceptions

Ah forgot to check for all places, ok did that

squashed commits


Reproducibility

Thank you for a great work.
I would like to reproduce the results and compare to another engine. Following the documentation, it appears that you have already included a test in ann-benchmarks:
The code to run OG-LVQ can be found here, and it was included in the ANN-benchmarks and Big-ANN-benchmarks evaluation codes following their guidelines.
Unfortunately, I am not able to find it. Could you please provide a link?
thank you

Thank you for your interest!
For our results, we made internal forks of the ANN-benchmarks and Big-ANN-benchmarks frameworks, as we collect more metrics than those in the original versions. As these forks are not part of SVS proper, we never got to release them. We're working on releasing sample code for the integration with these frameworks.

+1 My team would also be interested when you release code for the integration with ANN-benchmarks framework.

I am working on releasing the integration code. I will update this issue as I make progress.


Adds an explanation for environment variables

As requested by a user of the #prisma2-preview on Slack there's now an explanation for how to use environment variables and it should be easy to fit in an explanation for how .env files work when they are implemented.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

I always have an error that I have to define the other env variable, i think its a bug
https://gist.github.com/yovanoc/1a0b0d34ff4e8b22dc39cd36255c1f6d

Thanks @ChristianJacobsen ! @nikolasburk will look into your PR.
@yovanoc please create a separate issue for that, that would help us a lot. Thanks!

@timsuchanek Just done it https://github.com/prisma/prisma2/issues/221 thanks


how to convert sparse model to dense model?

After finishing pretrain resnet50, we can get a resnet50 weight file in sparse type. I'd like to use resnet50 (dense type) as my backbone in my other projects. But how to convert sparse model to dense model?  Is there any convenient function like SparseEncoder.dense_model_to_sparse in encoder.py ?

Actually we don't need a sparse-to-dense conversion. You can use the pretrained weight directly (just like an ImageNet-supervised pretrained resnet50) like the following code:
import torch, timm
res50 = timm.create_model('resnet50')
state = torch.load('resnet50_1kpretrained_timm_style.pth', 'cpu')
missing_keys, unexpected_keys = res50.load_state_dict(state, strict=False)
assert missing_keys == ['fc.weight', 'fc.bias']
assert len(unexpected_keys) == 0

The reason is we use torch builtin operators (nn.Conv2d, nn.LayerNorm, etc.) to simulate those sparse operators (in /pretrain/encoder.py). So it keeps the same weight formats and names.

@keyu-tian Sorry to bother you. From your way of loading the model, does it mean that SparK only uses MAE during training, does not change the original model structure, such as 'resnet', on inference? Looking forward to your reply, thank you!


[Feat] Update ItemWrapper

Update the placeholder item wrapper to show information about the item and it's actions.
This includes the following:

[x] Include a description for items and actions.
[ ] Include a mechanical summary for items and actions.
[x] Generate an action summary and buttons If only one action is present.
[x] Generate sub wrappers for actions if multiple actions exist.

Parent issue: #46

Closing in favour of more specific issues.


(BOLT-227) Catch wrong password and fail the node

If an incorrect password is provided for privilege escalation, the
channel previously blocked as it didn't identify the request to retry.
Add handling to identify retry after incorrect password and error on the
node (the password's not going to "get better").

I think this is safe as-is now, it will be caught by higher level StandardError handling and turned into a Bolt::Result.

Or should these all use something more like Bolt::Node::ConnectError or a new Bolt::Node::ExecuteError?

I plan to work on an acceptance test for this that

sets up a user (adduser, passwd)
run-as root from user, failure cases: no sudo-password supplied, sudo-password wrong, sudo-password right but no sudoers permission
run-as root from user, success case: sudoers permitted
run-as user from root, success case: no password needed

I have no idea why that test is failing :(

Fixed up. Did all the tests except a user without sudoers permission in spec tests.

https://github.com/puppetlabs/bolt/pull/174


Plugins v4 docs

used the description from the pr
docs :)

closes #240

I think this is the first time docs got merged before actual PR :poggies:


Trainer fails when Scan fails

Lately, I'm only able to have trainer run successfully when scan succeeds and passes all tests. If even one test fails, I get the following error:
    Unit tests failed

Followed by the sad explosion icon next to trainer and the fastlane finished with errors message.
I wanted to look into this so I saw there was a commit 7 days ago, the only instance of the message I could find the following, from the trainer_action.rb file, lines 11-13:
resulting_paths.each do |path, test_successful|
    UI.test_failure!("Unit tests failed") unless test_successful
end

I can provide any working examples you might need to reproduce this. I'm very stumped after an hour or so of searching for the solution. If my tests pass, trainer runs. If any tests fail, trainer kills the lane.

Are you sure this wasn't always the case? The only change that was recently applied was https://github.com/KrauseFx/trainer/commit/002f5d67299ea77de806f8482a59760468ff92c4

Actually no, I'm not confident that this is recent. Looking through my Fastfile history I had broken trainer out into another lane previously that was actually being left before reaching it, so it's entirely possible the issue has been around for longer.
This is a bug in trianer though, right?

@KrauseFx I just wanted to bring this back up, because I'd love to use Trainer with my lane. I have the same problem with it failing and have to turn it off in order for everything to work. My scan command has fail_build: false, then I run trainer so it fails and shows the little 💥 mark in the fastlane summary next to trainer having run for 0ms. Bottom of script says:
[22:03:24]: fastlane finished with errors

[!] Unit tests failed
script returned exit code 1

I'm not doing anything other than having a failing test with scan and trainer in the lane. If you need more information, I can try to get a test project.

We are also experiencing the same issue. It seems that even when fail_build: true is set (as seen from output), the whole build is flagged as failure and fastlane test exits with status 1:
[15:15:38]: ------------------
[15:15:38]: --- Step: scan ---
[15:15:38]: ------------------

< Xcode output omitted>

** TEST EXECUTE FAILED **
[15:16:20]: Exit status: 65
+--------------------+----+
|      Test Results       |
+--------------------+----+
| Number of tests    | 11 |
| Number of failures | 1  |
+--------------------+----+

[15:16:20]: ---------------------
[15:16:20]: --- Step: trainer ---
[15:16:20]: ---------------------

+------------------+---------------------------------------------------------------------------+
|                                  Summary for trainer 0.8.0                                   |
+------------------+---------------------------------------------------------------------------+
| output_directory | test                                                                      |
| fail_build       | false                                                                     |
| path             | DerivedData/Logs/Test/Test-Wire-iOS-2018.10.04_15-15-56-+0200.xcresult/T  |
|                  | estSummaries.plist                                                        |
| extension        | .xml                                                                      |
+------------------+---------------------------------------------------------------------------+

[15:16:20]: `Helper.gem_path` is deprecated. Use the `ROOT` constant from the appropriate tool module instead.
Successfully generated 'test/TestSummaries.xml'
+----------------------------+---------------------------------------------------------------------------+
|                                              Lane Context                                              |
+----------------------------+---------------------------------------------------------------------------+
| DEFAULT_PLATFORM           | ios                                                                       |
| PLATFORM_NAME              | ios                                                                       |
| LANE_NAME                  | ios test                                                                  |
| SCAN_DERIVED_DATA_PATH     | DerivedData                                                               |
| SCAN_GENERATED_PLIST_FILES | ["DerivedData/Logs/Test/Test-Wire-iOS-2018.10.04_15-15-56-+0200.xcresult  |
|                            | /TestSummaries.plist"]                                                    |
| SCAN_GENERATED_PLIST_FILE  | DerivedData/Logs/Test/Test-Wire-iOS-2018.10.04_15-15-56-+0200.xcresult/T  |
|                            | estSummaries.plist                                                        |
+----------------------------+---------------------------------------------------------------------------+
[15:16:20]: Unit tests failed

+------+------------------+-------------+
|           fastlane summary            |
+------+------------------+-------------+
| Step | Action           | Time (in s) |
+------+------------------+-------------+
| 1    | opt_out_usage    | 0           |
| 2    | default_platform | 0           |
| 3    | scan             | 41          |
| 💥   | trainer          | 0           |
+------+------------------+-------------+

[15:16:20]: fastlane finished with errors

[!] Unit tests failed

$> echo $status
1

We have the same issue with 2.108.0 and fail_build set to false
[15:29:43]: Unit tests failed

+------+--------------------+-------------+
|            fastlane summary             |
+------+--------------------+-------------+
| Step | Action             | Time (in s) |
+------+--------------------+-------------+
| 1    | xcversion          | 0           |
| 2    | clear_derived_data | 15          |
| 3    | scan               | 483         |
| 💥   | trainer            | 0           |
+------+--------------------+-------------+

[15:29:43]: fastlane finished with errors

[!] Unit tests failed


[TASK] Add ShellCheck / TravisCI

ShellCheck is a must have! Even better if integrated with TravisCI.

Added Shellcheck Github action with https://github.com/benweidig/portal/commit/5f9e3164e5d35366b3062454463e7f8e7117c3ac


HTTP Delete multiple item spec & standardization

There are multiples ways to do deletions on multiples items on an API.
Restful
HTTP route: DELETE /items&ids=id1,id2,id3
Pro: This is the rest way
Cons: There is a limit on the maximum number of items that can be deleted. (https://tools.ietf.org/html/rfc7230#section-3.1.1)
This is not possible !
1. Restful alternative
HTTP route: POST /items/delete-xxx body: [id1,id2,id3]
Pro: Infinite limit of items to delete. Not restricted to delete by id, it's possible to write a more complex structure. This used by some projects.
Cons: Could be considerate as ugly
e.g.
DELETE /stop-words/:id //delete ONE stop-word
DELETE /stop-words //delete ALL stop-words
POST /stop-words/delete-selection : //delete a selection of stop-word passed in the body
or POST /stop-words/delete
or POST /stop-words/delete-batch

2. Body inside DELETE
HTTP route: DELETE /items body: [id1,id2,id3]
Pro: Infinite limit of items to delete. Not restricted to delete by id, it's possible to write a more complex structure.
Cons: controversy about putting a body in a delete. RFCs don't specify anything (https://tools.ietf.org/html/rfc7231#section-4.3.5).
e.g.
DELETE /stop-words/:id //delete ONE stop-word
POST /stop-words/clear //delete ALL stop-words or
DELETE /stop-words/all //delete ALL stop-words
DELETE /stop-words : //delete a selection of stop-word passed in the body

There is something invalid with the second solution: there is a conflict between the DELETE /stop-words/:id and the DELETE /stop-words/all routes. The POST /stop-words/clear route is neither valid because it is not self-explanatory, what tells de user that it will delete all the stop-words as it is a POST ?

After reading others blogs/posts, I have a name alternative for the numero 1:
delete-batch instead of delete-selection. Maybe more "English". I don't know what you prefer (if you like the num 1 option).

I prefer the 1. with the POST /stop-words/delete-batch route.

Same as kero, the 1 with POST /stop-words/delete-batch

I would go with the first alternative with POST /stop-words/delete-batch

Other alternative :
`GET /stop-words` => ['one',two']
`POST /stop-words` body :["the","and"] => overrides ['one',two']
`DELETE /stop-words` => 204 No Content (deleted all stop words)

@bidoubiwa, pretty sure that's the current behavior on master. Am I wrong?

There is something invalid with the second solution: there is a conflict between the DELETE /stop-words/:id and the DELETE /stop-words/all routes, what if there is a stop-word "all"?
The POST /stop-words/clear route is neither valid because it is not self-explanatory, what tells de user that it will delete all the stop-words as it is a POST?

This is not true if we only use resources_ids. And if there is no id resource, it is probably because the complete entity is the resource?!

I don't understand, in the DELETE /documents/:id route couldn't :id corresponds to "all" if a user wants to set this word as a stop-word for example? This will collide with this second route DELETE /documents/all?

wa, pretty sure that's the current behavior on master (not on 8.4). Am I wrong?

Currently we have:
GET /stop-words => ['one',two']
PATCH /stop-words body :["the","and"] => overrides ['one',two']
POST /stop-words => 204 No Content (deleted all stop words)

@curquiza No, for the moment it is like this
PATCH /stop-words and the delete route is POST /stop-words
I want the delete route to be delete and to delete everything,
and the post route to be ONE ressource so it overrides the previous stop-words.

@qdequele  For me the resource_id of a stop-word is the stop-word itself. The stop-word is unique.

If the complete entity is the ressource than this would be the rest way :
GET /stop-words => ['one',two']
POST /stop-words body :["the","and"] => overrides ['one',two']
DELETE /stop-words => 204 No Content (deleted all stop words)

I assume that a stop-word is not an id but just a word in a array. This array is the global resource. Maybe it's a misunderstanding.


Make Roles enum an IntEnum subclass

Overview
While #73 was in review, #72 introduced a Roles enum. It wasn't particularly attractive to work with because it didn't subclass IntEnum. Now, it's an IntEnum, and it's used where possible.
Follow-up to https://github.com/azavea/iow-boundary-tool/pull/73#discussion_r975662996.
Testing Instructions

[x] scripts/resetdb should succeed

[ ] scripts/createsuperuser should succeed

scripts/manage shell

Run these commands and verify full_clean() still emits a ValidationError as show:

from api.models import User
c1 =<EMAIL_ADDRESS>c1.utilities.set([])
c1.full_clean()

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 1251, in full_clean
    raise ValidationError(errors)
django.core.exceptions.ValidationError: {'__all__': ['Contributors must be assigned a utility.']}

Checklist

[ ] fixup! commits have been squashed
[ ] CHANGELOG.md updated with summary of features or fixes, following Keep a Changelog guidelines
[ ] README.md updated if necessary to reflect the changes
[ ] CI passes after rebase

A small thing in the PR instruction:

scripts/createsuperuser should succeed

We don't have a separate createsuperuser script in this project and usually  ./scripts/manage createsuperuser will suffice here.

Thanks for taking a look!


MXKRoomViewController: Add option to custom scroll on keyboard presen…

…tation.
'scrollHistoryToTheBottomOnKeyboardPresentation': Tell whether the room history is automatically scrolled to the most recent messages when a keyboard is presented.

LGTM


30 times slower than jedis

I'm trying to migrate old jedis code to redisson and getting some weird result.
Roughly same code, jedis:
pool = new JedisPool(new JedisPoolConfig(), host, port, 20_000)

Later in code, from multiple threads:
try (Jedis j  = pool.getResource()) {
   v = j.hget(mapName, mapKey);
}

Redisson:
Config config = new Config()
config.useSingleServer()
      .setAddress(address)
      .setConnectTimeout(20_000)
      .setTimeout(20_000);

  RedissonClient c = Redisson.create(config);

Later in code, from multiple threads:
  RMap<byte[], byte[]> map = c.getMap(mapName, new ByteArrayCodec())
  v = map.get(mapKey)

Measurements 95% of queries:

Am I doing something wrong with redisson?

which version are you trying?

redisson 3.5.1, java 1.8.0_144, redis 4.0.1, debian 9.1

Please try 3.5.0 version due to this regression bug https://github.com/redisson/redisson/issues/1031
What is connections amount for jedis you use?

Based on CLIENT LIST, jedis version ~7-10 connections, redission version 60-70 connections probably.

what is the average size of objects stored in map?

also try to set Config.useSingelServer().setConnectionPoolSize(10)

I have just released 3.5.2 version. Give it a try too

Average size of object is about 5-10mb.
Seems like upgrade to 3.5.2 helped:

You have just updated a version, right? or did you change pool size to 10?

Just update was enough, thank you for help. Can you also comment this https://gitter.im/mrniko/redisson, should i open bug?

@valodzka I have answered. You could try PRO if you wanted to be faster :)

We're facing slowness issues as well:
We use Redis for caching. Each cached object is about 90 KB in size. We set the Redisson timeout for 250 ms. Using JMeter, when more than 10-15 users simultaneously access a web page, which we use Redis to render, we're getting many timeouts from Redisson, and each get request takes much more than numerous ms, which is what I'd expect.
I tried running JMeter from another machine at the same time, to see if it slows down Redisson even more, but it didn't seem to make a difference, so I assume it doesn't have to do with the Redis server, but with Redisson.
To solve this, following this thread, we upgraded to the latest Redisson and changed all the definitions according to what's defined here, with no success.
Can you think of another reason for this?
Thanks

@dekelbz
Could you share the code of your Redis based cache implementation ?

Thanks for the quick response.
Here's my current code:
private RedissonClient getRedissonClient() {         Config config = new Config();         config.setCodec(new SnappyCodec())                 .useSingleServer()                 .setAddress(redisUrl)                 .setPassword(redisPassword)                 .setTimeout(TIMEOUT);         try {             return Redisson.create(config);         } catch (Exception e) {             LOG.error("error while connecting to Redis", e);             return null;         }     }
Previously I tried adding the configurations mentioned here and some more (all together like what I pasted, and also any combination
private RedissonClient getRedissonClient() {         Config config = new Config();         config.setCodec(new SnappyCodec())                 .useSingleServer()                 .setAddress(redisUrl)                 .setPassword(redisPassword)                 .setTimeout(TIMEOUT)                 .setTcpNoDelay(true)                 .setSubscriptionsPerConnection(1024)                 .setConnectionPoolSize(1024);         try {             return Redisson.create(config);         } catch (Exception e) {             LOG.error("error while connecting to Redis", e);             return null;         }     }

Which Redisson objects do you use for cache access?

I use RedissonClient. The .get() is what takes very long

@dekelbz Have you tried to test it locally?


Call the correct application module start fun

The application name and start module is not always the same. This PR takes this into account.

Wild goose chase - as a convention we always add the custom start function on the 'appname.erl' file and not in the 'appname_app.erl' file.


Added healthcheck for container and logging

Going ahead and merging this, please review when you get a chance


Write in README.md the precise versions of the supported browsers

Following https://github.com/nhn/tui.calendar/pull/618, write in README.md minimal versions of the supported browsers.  This shall make possible to determine, which JavaScript (and CSS) functions can be used directly without polyfills: Function.bind, Array.indexOf()…

This is our team's policy.
Except for ie, it will change all browsers to "latest".


Fix test in Edge

Edge renders attributes in differing order.

Given there were already two orderings being tested against, I figure this issue has come up before.
If a different ordering comes up in the future, we may want to just add all six permutations of the attribute order to this test.


Change app name to monitor-app

Purpose

Previous app name example-siddhi-deployment made the reader confused. So, change that name to monitor-app.

Goals

Remove confusions.

Approach

Change kubernetes section of the documentation accordingly.

Test environment

minikube version: v0.33.1

Codecov Report

Merging #1226 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #1226      +/-   ##
============================================
+ Coverage     61.45%   61.46%   +0.01%
  Complexity       35       35
============================================
  Files           619      619
  Lines         24942    24942
  Branches       3909     3909
============================================
+ Hits          15327    15331       +4
+ Misses         8288     8284       -4
  Partials       1327     1327

Impacted Files
Coverage Δ
Complexity Δ

...ut/stream/state/AbsentStreamPreStateProcessor.java
85.21% <0%> (-1.41%)
0% <0%> (ø)

.../siddhi/core/util/event/handler/StreamHandler.java
74% <0%> (+12%)
0% <0%> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eefec78...fafa7df. Read the comment docs.

@suhothayan Need ur help to republish the content to the site


Improve speed of ConsGenIncProcessModel solvers

The ConsGenIncProcessModel solvers, with default values, are currently the longest running automated tests.
Maybe there is a way to improve the performance of these solvers.

Why not just have them only solve for a few periods?  Just set cycles=1 or
5 or something.
On Thu, Mar 12, 2020 at 5:46 PM Sebastian Benthall<EMAIL_ADDRESS>wrote:

The ConsGenIncProcessModel solvers, with default values, are currently the
longest running automated tests.
Maybe there is a way to improve the performance of these solvers.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFMEFDIRJ5XTDLIGWE3RHFJ2RANCNFSM4LGV5CGQ
.

Hmmm. Ok, I'm trying that,
So, with cycles=1, it looks like the solver does not make much progress on converging on a solution.
With cycles=5, I get errors in the updatepLvlGrid method, because of an assertion the cycles equal either 0 or 1.
https://github.com/econ-ark/HARK/blob/d1a8eb90ebe34a93801ead8655b5e29ef9d1e3eb/HARK/ConsumptionSaving/ConsGenIncProcessModel.py#L1142

I guess I'll go with cycles=1.
I wonder if you could help me understand what's going on here though.
Why is does cycles=1 mean that the model is a lifecycle model?

Sure.  The cycles attribute indicates how many times the sequence of
periods described by the time-varying parameters will "happen" to the
agent, with 0 standing in for infinity.  For any value of cycles other than
0, it's a finite horizon or lifecycle model-- there is some definite end
point that the agent cannot exist past.  Basically every model that
everyone ever works with would be specified as either cycles=0 or cycles=1;
having cycles >= 2 isn't completely useless, but it's best used for
teaching (or sometimes debugging).  For example, you can quickly plot how
an infinite horizon solution converges by setting cycles=50 and solving a
problem with a 1 period cycle (the policy function visually converges in
about 50 periods).
The default dictionary for ConsGenIncProcess model specifies a cycle that
is T_cycles=1 periods long.  If you set cycles=0, the model solves the
problem of "what should I do if I will live out this kind of period over
and over again?".  If you set cycles to 1 with a one period cycle, you are
asking it to solve the problem of "what should I do if I live this period,
and then it's the terminal period"?  The solution isn't trying to converge,
you're just asking it to solve a problem with one non-terminal period and
one (trivial) terminal period.  It's a lifecycle model for a mayfly.
Setting cycles=5 works fine for most models in HARK, but
ConsGenIncProcess is a special case because of the pLvlGrid.  This
attribute is constructed in one of the methods called by update; it
represents the grid of permanent income levels at which to solve for
optimal consumption, with the consumption function interpolated in
between.  It's constructed by simulation, literally drawing out the
population of pLvl at each time index of the cycle and then grabbing
percentiles as given in the primitive attribute pLvlPctiles.  In the
default dictionary, it captures the middle 99.8% of the distribution.  If
cycles=1, this just means looping over each period of the lifecycle and
finding the distribution of permanent income at each age.  If cycles=0, the
code assumes that 1000 periods is a "long time" and simulates the
population for that long, then uses the distribution of pLvl in that
fake-ergodic distribution at each t of the cycle.
But if cycles >= 2, this method won't work.  The distribution of pLvl is
different on successive cycles; there is not a one-to-one mapping between
t_age and t_cycle, nor is there a meaningful "long run" distribution.
There are ways around this, but I didn't take a stand on how we should
handle this non-standard case when I wrote that method.  As is, if you
really wanted to have a sequence of periods (or just one period) repeat a
finite number of times, you would have to write the parameter dictionary to
have that loop in every time-varying primitive parameter, and then set
cycles=1.  For example, PermGroFac would change from being [1.02] to
5*[1.02].
On Fri, Mar 13, 2020 at 12:35 PM Sebastian Benthall <
<EMAIL_ADDRESS>wrote:

I guess I'll go with cycles=1.
I wonder if you could help me understand what's going on here though.
Why is does cycles=1 mean that the model is a lifecycle model?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567#issuecomment-598810289, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFIIXBZB2H2UXZK7IE3RHJOFVANCNFSM4LGV5CGQ
.

Thanks for explaining all that.
So if I understand correctly...

T_cycles is the number of periods in a cycle, a property of the model
cycles is the number (probably infinite or 1) of times the agent lives through the cycle, for the purpose of the solver
T_sim is the number of periods to be simulated when simulate() is called

And ConsGenIncProcess doesn't use dynamic programming in the solver in the same way as the other models, but uses simulation internally, with a 1000 period limit.
I've made some changes to #527 that set cycles=1 for the tests, but these tests at this point just make sure the code doesn't break; it doesn't get at the internals at all.
But maybe there is a way to improve the speed of the ConsGenIncProcessModel solver by introducing a threshold that would interrupt the solver's internal simulation if the model converges before 1000 periods. It sounds like 50 is sufficient to prove the concept?

Not quite.  Both T_cycle and cycles are instance-level attributes.  The
model that an AgentType subclass solves is agnostic to T_cycle and cycles;
the specific instance of that model (a "problem") fills in those
time-defining attributes.
T_sim as an attribute represents the maximum number of periods you would
ever be interested in simulating for this instance.  It's used to
initialize the arrays that will track idiosyncratic variable histories of
the attributes named in track_vars.  When simulate() is called, it will
simulate for T_sim period by default, but you can also specify an argument
T <= T_sim for it to simulate for.  This is useful for doing things like
"simulate 100 periods, then screw with the state of the world, then
simulate 300 more periods".
ConsGenIncProcess uses backward iteration methods to solve the model, just
like all our other solvers.  It simply uses simulation to construct the
grid the problem will be solved on, in the pre-solution step.  The 1000
periods thing is just an approximation to the "long run" distribution of
permanent income.  I.e. if a model period is a year, it assumes that after
1000 years, the population distribution of pLvl is roughly distributed the
same as it would after 10,000 years or 10B years.
"Making sure the code doesn't break" is a really, really important test.
Having the solver try a 1 or 2 period problem and then check to see if a
few points of the policy function match the solution from the prior version
of the code is pretty much the best we can do with our model tests.
Your last paragraph is incorrectly premised for the reasons above.
On Fri, Mar 13, 2020 at 2:33 PM Sebastian Benthall<EMAIL_ADDRESS>wrote:

Thanks for explaining all that.
So if I understand correctly...

T_cycles is the number of periods in a cycle, a property of the
model
cycles is the number (probably infinite or 1) of times the agent
lives through the cycle, for the purpose of the solver
T_sim is the number of periods to be simulated when simulate() is
called

And ConsGenIncProcess doesn't use dynamic programming in the solver in
the same way as the other models, but uses simulation internally, with a
1000 period limit.
I've made some changes to #527 https://github.com/econ-ark/HARK/pull/527
that set cycles=1 for the tests, but these tests at this point just make
sure the code doesn't break; it doesn't get at the internals at all.
But maybe there is a way to improve the speed of the
ConsGenIncProcessModel solver by introducing a threshold that would
interrupt the solver's internal simulation if the model converges before
1000 periods. It sounds like 50 is sufficient to prove the concept?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567#issuecomment-598858948, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFJDEP5TJGGWNCTNBLTRHJ36XANCNFSM4LGV5CGQ
.

Also, I just noticed that our documentation notebooks are in the
/examples folder (at least on the branch I'm working on), and the actual
examples have been deleted and replaced with bizarro notebook output.  Is
this intended?  I thought the whole point of the /examples directory is to
have small, minimal .py files that implement our models?
On Fri, Mar 13, 2020 at 2:48 PM Matthew White<EMAIL_ADDRESS>wrote:

Not quite.  Both T_cycle and cycles are instance-level attributes.  The
model that an AgentType subclass solves is agnostic to T_cycle and cycles;
the specific instance of that model (a "problem") fills in those
time-defining attributes.
T_sim as an attribute represents the maximum number of periods you would
ever be interested in simulating for this instance.  It's used to
initialize the arrays that will track idiosyncratic variable histories of
the attributes named in track_vars.  When simulate() is called, it will
simulate for T_sim period by default, but you can also specify an argument
T <= T_sim for it to simulate for.  This is useful for doing things like
"simulate 100 periods, then screw with the state of the world, then
simulate 300 more periods".
ConsGenIncProcess uses backward iteration methods to solve the model, just
like all our other solvers.  It simply uses simulation to construct the
grid the problem will be solved on, in the pre-solution step.  The 1000
periods thing is just an approximation to the "long run" distribution of
permanent income.  I.e. if a model period is a year, it assumes that after
1000 years, the population distribution of pLvl is roughly distributed the
same as it would after 10,000 years or 10B years.
"Making sure the code doesn't break" is a really, really important test.
Having the solver try a 1 or 2 period problem and then check to see if a
few points of the policy function match the solution from the prior version
of the code is pretty much the best we can do with our model tests.
Your last paragraph is incorrectly premised for the reasons above.
On Fri, Mar 13, 2020 at 2:33 PM Sebastian Benthall <
<EMAIL_ADDRESS>wrote:

Thanks for explaining all that.
So if I understand correctly...

T_cycles is the number of periods in a cycle, a property of the
model
cycles is the number (probably infinite or 1) of times the agent
lives through the cycle, for the purpose of the solver
T_sim is the number of periods to be simulated when simulate() is
called

And ConsGenIncProcess doesn't use dynamic programming in the solver in
the same way as the other models, but uses simulation internally, with a
1000 period limit.
I've made some changes to #527
https://github.com/econ-ark/HARK/pull/527 that set cycles=1 for the
tests, but these tests at this point just make sure the code doesn't break;
it doesn't get at the internals at all.
But maybe there is a way to improve the speed of the
ConsGenIncProcessModel solver by introducing a threshold that would
interrupt the solver's internal simulation if the model converges before
1000 periods. It sounds like 50 is sufficient to prove the concept?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567#issuecomment-598858948, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFJDEP5TJGGWNCTNBLTRHJ36XANCNFSM4LGV5CGQ
.

Nevermind, I found it one folder down.
Open up examples/ConsumptionSaving/example_ConsGenIncProcessModel.py ... At
lines 38 and 130 you can add the argument True to the call to solve() if
you want to see it run cycle by cycle.  These examples only take about 6-8
seconds to solve; this doesn't seem like a long time to me.
Also, it looks like process_time is giving weird numbers.  Can we use
time.time instead?
On Fri, Mar 13, 2020 at 2:56 PM Matthew White<EMAIL_ADDRESS>wrote:

Also, I just noticed that our documentation notebooks are in the
/examples folder (at least on the branch I'm working on), and the actual
examples have been deleted and replaced with bizarro notebook output.  Is
this intended?  I thought the whole point of the /examples directory is to
have small, minimal .py files that implement our models?
On Fri, Mar 13, 2020 at 2:48 PM Matthew White<EMAIL_ADDRESS>wrote:

Not quite.  Both T_cycle and cycles are instance-level attributes.  The
model that an AgentType subclass solves is agnostic to T_cycle and cycles;
the specific instance of that model (a "problem") fills in those
time-defining attributes.
T_sim as an attribute represents the maximum number of periods you would
ever be interested in simulating for this instance.  It's used to
initialize the arrays that will track idiosyncratic variable histories of
the attributes named in track_vars.  When simulate() is called, it will
simulate for T_sim period by default, but you can also specify an argument
T <= T_sim for it to simulate for.  This is useful for doing things like
"simulate 100 periods, then screw with the state of the world, then
simulate 300 more periods".
ConsGenIncProcess uses backward iteration methods to solve the model,
just like all our other solvers.  It simply uses simulation to construct
the grid the problem will be solved on, in the pre-solution step.  The
1000 periods thing is just an approximation to the "long run" distribution
of permanent income.  I.e. if a model period is a year, it assumes that
after 1000 years, the population distribution of pLvl is roughly
distributed the same as it would after 10,000 years or 10B years.
"Making sure the code doesn't break" is a really, really important test.
Having the solver try a 1 or 2 period problem and then check to see if a
few points of the policy function match the solution from the prior version
of the code is pretty much the best we can do with our model tests.
Your last paragraph is incorrectly premised for the reasons above.
On Fri, Mar 13, 2020 at 2:33 PM Sebastian Benthall <
<EMAIL_ADDRESS>wrote:

Thanks for explaining all that.
So if I understand correctly...

T_cycles is the number of periods in a cycle, a property of the
model
cycles is the number (probably infinite or 1) of times the agent
lives through the cycle, for the purpose of the solver
T_sim is the number of periods to be simulated when simulate()
is called

And ConsGenIncProcess doesn't use dynamic programming in the solver in
the same way as the other models, but uses simulation internally, with a
1000 period limit.
I've made some changes to #527
https://github.com/econ-ark/HARK/pull/527 that set cycles=1 for the
tests, but these tests at this point just make sure the code doesn't break;
it doesn't get at the internals at all.
But maybe there is a way to improve the speed of the
ConsGenIncProcessModel solver by introducing a threshold that would
interrupt the solver's internal simulation if the model converges before
1000 periods. It sounds like 50 is sufficient to prove the concept?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567#issuecomment-598858948,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFJDEP5TJGGWNCTNBLTRHJ36XANCNFSM4LGV5CGQ
.

@sbenthall please confer with @MridulS who wrote the excellent configurator.py tool for trying out various configurations of parameter values for the CGMPortfolio REMARK.  This was intended as a prototype for a more general purpose tool that I hope we can construct that will define a standard procedure for configuring and running our models.  That way, there could be a special configuration for DemARK testing that, for example, had just a 5 or 10 period configuration for ConsGenIncProcessModel instead of its default.  We could have special "error-testing" configurations for all DemARKs (and for that matter REMARKs) that would be constructed to run fast while still doing at least a bit of a workout of the code.

As written, no. The idea that 1000 periods is a "long time" in model terms
is hard coded. It's not pulled from an attribute.
That could be changed with 2 lines, but there's not really a point. That
construction step takes 1 second or so, probably much less. So there isn't
really a point in speeding it up for testing purposes.  It's literally just
drawing 10,000,000 or so lognormal shocks, executing 10,000,000
multiplications, sorting 10,000 numbers, and pulling a few percentiles.
This is not a burdensome step.
On Fri, Mar 13, 2020, 7:19 PM Sebastian Benthall<EMAIL_ADDRESS>wrote:

Keeping to the original question:
ConsGenIncProcess uses backward iteration methods to solve the model, just
like all our other solvers. It simply uses simulation to
construct the grid the problem will be solved on, in the pre-solution
step. The 1000
periods thing is just an approximation to the "long run" distribution of
permanent income. I.e. if a model period is a year, it assumes that after
1000 years, the population distribution of pLvl is roughly distributed the
same as it would after 10,000 years or 10B years.
So, suppose one wanted to write a test for this functionality, but did
not want to wait for it to go through 1000 steps.
Maybe there is or could be a way to limit that number of steps to 50, for
the purpose of testing.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567#issuecomment-598969671, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFK4DM7LENYREOCWILTRHK5PHANCNFSM4LGV5CGQ
.

Ah, I was under the impression that this simulation step was the performance bottleneck.
I may be mistaken.
The next thing I'll try is performance profiling tools to see what's slowing the solver down.

How long is the solver taking for you? It should only take a few seconds if
it's running the default dictionary.
Any profiler will tell you that the most expensive line is whichever one
evaluates vPfuncNext. That's where literally 95% of any computation will
be, as it's evaluating a linear interpolation over 1D interpolators for
about 485624 realizations of mNrmNext (if I remember gridsizes in the
default dictionary correctly).
On Fri, Mar 13, 2020, 9:09 PM Sebastian Benthall<EMAIL_ADDRESS>wrote:

Ah, I was under the impression that this simulation step was the
performance bottleneck.
I may be mistaken.
The next thing I'll try is performance profiling tools to see what's
slowing the solver down.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567#issuecomment-598990572, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFPPCKODVDTQDHV4ULDRHLKLTANCNFSM4LGV5CGQ
.

Maybe there is or could be a way to limit that number of steps to 50, for the purpose of testing.

Another use case for the comparator-type tool that @MridulS is working on.

So, suppose one wanted to write a test for this functionality, but did not want to wait for it to go through 1000 steps.

Actually, there are a number of potential tests.  These notes  derive a number of propositions about the long run distribution of a process like
$p_{t+1} = \gamma + \phi p_{t} + \epsilon_{t+1}$
the most important of which is probably:
$\sigma^{2}{p} = \left(\frac{\phi}{1-\phi^{2}}\right) \sigma^{2}{\epsilon}$
and another of which is that $p$ itself should be distributed according to
$p \sim N\left(\left(\frac{\gamma}{1-\phi}\right),\sigma^{2}_{p}\right)$
Actually, it might be better simply to construct this distribution at the outset and then see whether after some number of periods of simulation (50?) the distribution's mean and variance are "close enough" to the analytical values (where "close enough" would be, say, only one chance in 10000 that the mean of the resulting distribution would be farther away than $x$ from its analytical value.  If the number of periods is $n$ then the process of simulating $n$ periods is to add $n$ normally distributed shocks to the process, and then to do a test of whether after those
shocks the data look like they are still normally distributed according to the
original, supposedly stationary, distribution.  (There are standard tests for
whether a distribution is normal with a given variance -- I think a $\chi^{2}$ test.)

We can put in tests like that, but they shouldn't go in
ConsGenIncProcessModel.  The point of this model is that (log) permanent
income can follow any process, not just p_{t+1} = \gamma + \phi_t +
\epsilon_{t+1} (random walk with drift; replacement/death events also
occur).  That special process is covered by ConsIndShockModel, and it
allows us to normalize out permanent income when solving the model.  And as
Chris notes, he has derived closed form solutions for the long run
distribution of p_t for that special process.  If anywhere, these kind of
tests should be built from that model.
ConsGenIncProcess exists to handle the generic case: P_{t+1} =
SomeFunction(P_t) * \psi_{t+1}.  Note the change in notation from lower
(logs) to capital (levels) for permanent income, and the switch from
\epsilon to \psi.  If we are completely ignorant about SomeFunction, then
there is no way to know what the long run distribution of P_t or p_t should
be, and thus no tests we should run.  There are special cases where this
can be derived, like an AR1 in logs, but there's no generic result.
On Sat, Mar 14, 2020 at 7:59 AM Christopher Llorracc Carroll <
<EMAIL_ADDRESS>wrote:

Maybe there is or could be a way to limit that number of steps to 50, for
the purpose of testing.
Another use case for the comparator-type tool that @MridulS
https://github.com/MridulS is working on.
So, suppose one wanted to write a test for this functionality, but did not
want to wait for it to go through 1000 steps.
Actually, there are a number of potential tests. These notes
http://halweb.uc3m.es/esp/Personal/personas/amalonso/esp/TSAtema4.pdf
derive a number of propositions about the long run distribution of a
process like
$p_{t+1} = \gamma + \phi p_{t} + \epsilon_{t+1}$
the most important of which is probably:
$\sigma^{2}{p} = \left(\frac{\phi}{1-\phi^{2}}\right) \sigma^{2}
{\epsilon}$
and another of which is that $p$ itself should be distributed according to
$p \sim N\left(\left(\frac{\gamma}{1-\phi}\right),\sigma^{2}_{p}\right)$
Actually, it might be better simply to construct this distribution at the
outset and then see whether after some number of periods of simulation
(50?) the distribution's mean and variance are "close enough" to the
analytical values (where "close enough" would be, say, only one chance in
10000 that the mean of the resulting distribution would be farther away
than $x$ from its analytical value. If the number of periods is $n$ then
the process of simulating $n$ periods is to add $n$ normally distributed
shocks to the process, and then to do a test of whether after those
shocks the data look like they are still normally distributed according to
the
original, supposedly stationary, distribution. (There are standard tests
for
whether a distribution is normal with a given variance -- I think a
$\chi^{2}$ test.)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567#issuecomment-599047808, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFKVX2HGIFYTKQUZPR3RHNWRVANCNFSM4LGV5CGQ
.

How long is the solver taking for you? It should only take a few seconds if it's running the default dictionary.

Running this test file:
https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/tests/test_ConsGenIncProcessModel.py
is taking 62.7 seconds on my machine.

It looks to me like those tests are equivalent to what's being run in
example_ConsGenIncProcess, but with a shorter simulation period.  Those two
examples take 9.2 seconds and 9.5 seconds to solve on my computer, and the
simulation should take about half a second (it takes 10 seconds to simulate
500 periods).
For all of our solvers, I recommend against testing them by solving an
infinite horizon problem.  We should set cycles to something like 2 or 5,
and target our tests off of that small-ish model.  These are tests of
whether the solver produces the same result after a code change.
On Sat, Mar 14, 2020 at 9:41 AM Sebastian Benthall<EMAIL_ADDRESS>wrote:

How long is the solver taking for you? It should only take a few seconds
if it's running the default dictionary.
Running this test file:
https://github.com/econ-ark/HARK/blob/master/HARK/ConsumptionSaving/tests/test_ConsGenIncProcessModel.py
is taking 62.7 seconds on my machine.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/econ-ark/HARK/issues/567#issuecomment-599062534, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ADKRAFJXURZ67H6LBT3GRLDRHOCQJANCNFSM4LGV5CGQ
.

Because GitHub markdown does not display math, and raw LaTeX code looks terrible, @mnwhite made a guess about what I was probably saying, but it's not what I was actually saying.  My math was for the specialization of the general income process that Matt mentions at the end of his comment: log p_{t+1} = \gamma + \phi \log p_{t} + \epsilon_{t+1}.  This would be a good default for simulating income dynamics because there is a closed form solution for the steady state distribution (per my math -- copy and paste into a markdown editor that can handle LaTeX and then it becomes readable!).

And as Chris notes, he has derived closed form solutions for the long run distribution of p_t for that special process.

Actually, for the special process with purely permanent shocks and a positive probability of death, the formula in cstwMPC is for the variance of the square of P; I don't have a formula for the steady state distribution (which is unbounded above but not lognormal).  But it would be easy enough to test whether the simulation produces something close to the derived variance.  (I did that when I originally derived the formula to make sure my math was right, and confirmed the result).

Oops, finger slipped and hit "close and comment" rather than just "comment".  Reopening.


Changing hours in day

Im trying to customise to use certain hours of the day (for example 9-5)
If i change the hours to reflect this. I can draw the grid etc fine
But rendering the current time and the events are not correct.
From my understanding, its due to the logic within the getRelativeTopInDay and getEventCellPositionStyle methods.
But i'm struggling to figure out the logic. Is there an easy way to make this universal based on the hours variable in utils.js?

Ok i figured it out for my use case.

How did you solve your problem? If my understanding is correct, you should set scrollOffsetMinutes.
Or, are you trying to narrow the hour range? That could be a future feature.


Move android-specific capabilities into their own package

To help developers understand what works where, should we create a package just for android-specific services (and maybe something similar for ios-specific services). ? Importing something like package:flutter_android/flutter_android.dart makes this very obvious, plus also helps with doc organization.
For example, specifically the activity field from the services library.
Thanks!

Fixed.  We don't have any android-specific capabilities now.