Cannot connect to port 25

Hi together. I have installed the docker-mailserver and everything worked fine for me but now I discovered that from the docker container I cannot connect to port 25 to other mail hosts and I cannot connect from other hosts to my server with the docker container on port 25. As a result I cannot send external mails and I cannot receive mails from other hosts.
When I add a rule with iptables on my server where the docker image is running:
iptables -D DOCKER -j ACCEPT -p tcp --dport 25 -d <IP_ADDRESS>
Then I can receive mails from outside. For internal mails everything works.
root@mail ~ # docker port mail
1525/tcp -> <IP_ADDRESS>:32786
587/tcp -> <IP_ADDRESS>:587
993/tcp -> <IP_ADDRESS>:993
143/tcp -> <IP_ADDRESS>:143

Does anybody had the same issues?

port 25 is not secure so for internal usage only ...  instead use port 587 for secure external access

I don't use iptables for my firewall (infrastructure firewall) but could you post your iptables?

First thanks a lot for your answers. I think I know the source if my problem. The docker proxy does not bind the ports how I want it.
root      8321  1114  0 19:08 ?        00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip <IP_ADDRESS> -host-port 993 -container-ip <IP_ADDRESS> -container-port 993
root      8332  1114  0 19:08 ?        00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip <IP_ADDRESS> -host-port 587 -container-ip <IP_ADDRESS> -container-port 587
root      8343  1114  0 19:08 ?        00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip <IP_ADDRESS> -host-port 143 -container-ip <IP_ADDRESS> -container-port 143
root      8354  1114  0 19:08 ?        00:00:00 /usr/bin/docker-proxy -proto tcp -host-ip <IP_ADDRESS> -host-port 32770 -container-ip <IP_ADDRESS> -container-port 1525

But in my configuration file for docker-compose it is:
  container_name: mail     ports:     - 143:143     - 587:587     - 993:993     - 25:25
I really hope somebody can help me.

Avoid port 25 when interacting with external hosts its obsolete and NOT secure
... instead use port 587 ... have you tried this ?
your-app on port 587 sends outgoing email to  -->  tomav/docker-mailserver listening on port 587  --><EMAIL_ADDRESS>

I use the port 587 to forward the mail to the docker host but then the docker-mailserver connects to port 25 on the other mail server to send it. When I send a mail from Google to my server there is a connection try on port 25.

Something else listening on port 25?
$ netstat -tapn

No, netstat shows nothing under port 25. It was also a fresh Debian installation.

@timotion83 You can use nc -l 25
and then telnet from a other box to it?
my $docker inspect mail
[
    {
        "Id": "xxx",
        "Created": "2017-08-12T17:30:42.870935388Z",
        "Path": "/bin/sh",
        "Args": [
            "-c",
            "supervisord -c /etc/supervisor/supervisord.conf"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 15843,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2017-08-12T17:30:43.896041747Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:0e57c61b84e866fe8768ab39fb0430226dc0cfb7e67aa1f153b3d137faf85995",
        "ResolvConfPath": "/srv/apps/docker/data/containers/e179e32d9689c1024b90af885579012f3c45d268769198686b76c81ff0579674/resolv.conf",
        "HostnamePath": "/srv/apps/docker/data/containers/e179e32d9689c1024b90af885579012f3c45d268769198686b76c81ff0579674/hostname",
        "HostsPath": "/srv/apps/docker/data/containers/e179e32d9689c1024b90af885579012f3c45d268769198686b76c81ff0579674/hosts",
        "LogPath": "/srv/apps/docker/data/containers/e179e32d9689c1024b90af885579012f3c45d268769198686b76c81ff0579674/e179e32d9689c1024b90af885579012f3c45d268769198686b76c81ff0579674-json.log",
        "Name": "/mail",
        "RestartCount": 0,
        "Driver": "aufs",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/srv/apps/certs:/etc/letsencrypt/live:rw",
                "/srv/apps/mail/data/mail-state:/var/mail-state:rw",
                "/srv/apps/mail/config:/tmp/docker-mailserver:rw",
                "/srv/apps/mail/data/mail:/var/mail:rw"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "apps_app_net",
            "PortBindings": {
                "143/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "143"
                    }
                ],
                "25/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "25"
                    }
                ],
                "4190/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "4190"
                    }
                ],
                "587/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "587"
                    }
                ],
                "993/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "993"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "always",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": [],
            "CapAdd": [
                "NET_ADMIN"
            ],
            "CapDrop": null,
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Data": null,
            "Name": "aufs"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/srv/apps/certs",
                "Destination": "/etc/letsencrypt/live",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/srv/apps/mail/data/mail-state",
                "Destination": "/var/mail-state",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/srv/apps/mail/config",
                "Destination": "/tmp/docker-mailserver",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "bind",
                "Source": "/srv/apps/mail/data/mail",
                "Destination": "/var/mail",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            }
        ],
        "Config": {
            "Hostname": "mail",
            "Domainname": "xxx.net",
            "User": "root",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "110/tcp": {},
                "143/tcp": {},
                "25/tcp": {},
                "4190/tcp": {},
                "587/tcp": {},
                "993/tcp": {},
                "995/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "ENABLE_SPAMASSASSIN=1",
                "ENABLE_MANAGESIEVE=1",
                "LDAP_BIND_DN=cn=admin,dc=xxx,dc=net",
                "ENABLE_SASLAUTHD=1",
                "LDAP_SERVER_HOST=ldap",
                "DMS_DEBUG=0",
                "DOVECOT_USER_FILTER=(&(objectClass=PostfixBookMailAccount)(|(uniqueIdentifier=%n)(mail=%u)))",
                "LDAP_QUERY_FILTER_ALIAS=(&(mailAlias=%s)(mailEnabled=TRUE))",
                "SASLAUTHD_LDAP_BIND_DN=cn=admin,dc=xxx,dc=net",
                "LDAP_QUERY_FILTER_USER=(&(mail=%s)(mailEnabled=TRUE))",
                "LDAP_SEARCH_BASE=dc=xxx,dc=net",
                "SSL_TYPE=letsencrypt",
<EMAIL_ADDRESS>                "SASLAUTHD_MECHANISMS=ldap",
                "LDAP_QUERY_FILTER_GROUP=(&(mailGroupMember=%s)(mailEnabled=TRUE))",
                "ENABLE_CLAMAV=1",
                "LETSENCRYPT_HOST=mail.xxx.net",
                "SASLAUTHD_PASSWD=",
                "LDAP_BIND_PW=xxx",
                "DOVECOT_PASS_FILTER=(&(objectClass=PostfixBookMailAccount)(|(uniqueIdentifier=%n)(mail=%u)))",
                "SASLAUTHD_LDAP_SEARCH_BASE=dc=xxx,dc=net",
                "SASLAUTHD_LDAP_PASSWORD=xxx",
                "VIRTUAL_HOST=mail..x.x.x.net",
                "SASLAUTHD_LDAP_PROTO=",
                "SASLAUTHD_LDAP_FILTER=(&(objectClass=PostfixBookMailAccount)(uniqueIdentifier=%U))",
                "ENABLE_FAIL2BAN=1",
                "LDAP_QUERY_FILTER_DOMAIN=(&(|(mail=*@%s)(mailalias=*@%s)(mailGroupMember=*@%s))(mailEnabled=TRUE))",
                "ENABLE_LDAP=1",
                "PERMIT_DOCKER=host",
                "SASLAUTHD_LDAP_SERVER=ldap",
                "ONE_DIR=1",
                "ENABLE_POSTGREY=0",
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "DEBIAN_FRONTEND=noninteractive",
                "VIRUSMAILS_DELETE_DELAY=7",
                "POSTGREY_DELAY=300",
                "POSTGREY_MAX_AGE=35",
                "POSTGREY_TEXT=Delayed by postgrey",
                "SASLAUTHD_MECH_OPTIONS="
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "supervisord -c /etc/supervisor/supervisord.conf"
            ],
            "ArgsEscaped": true,
            "Image": "apps_mail",
            "Volumes": {
                "/etc/letsencrypt/live": {},
                "/tmp/docker-mailserver": {},
                "/var/mail": {},
                "/var/mail-state": {}
            },
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "cea5e52be3a103a5897c3568655094a7d1c8f56e7584847646828109e7596eb8",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "apps",
                "com.docker.compose.service": "mail",
                "com.docker.compose.version": "1.13.0"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "xxx",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "110/tcp": null,
                "143/tcp": [
                    {
                        "HostIp": "<IP_ADDRESS>",
                        "HostPort": "143"
                    }
                ],
                "25/tcp": [
                    {
                        "HostIp": "<IP_ADDRESS>",
                        "HostPort": "25"
                    }
                ],
                "4190/tcp": [
                    {
                        "HostIp": "<IP_ADDRESS>",
                        "HostPort": "4190"
                    }
                ],
                "587/tcp": [
                    {
                        "HostIp": "<IP_ADDRESS>",
                        "HostPort": "587"
                    }
                ],
                "993/tcp": [
                    {
                        "HostIp": "<IP_ADDRESS>",
                        "HostPort": "993"
                    }
                ],
                "995/tcp": null
            },
            "SandboxKey": "/var/run/docker/netns/xxx",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "apps_app_net": {
                    "IPAMConfig": {
                        "IPv4Address": "172.x.x.x"
                    },
                    "Links": [
                        "ldap:ldap"
                    ],
                    "Aliases": [
                        "mail",
                        "e179e32d9689"
                    ],
                    "NetworkID": "b4a8f31854736cdf420b0c160b508e8333cfda6a4b5fc5afb161b9765a9b211e",
                    "EndpointID": "d205d0f3d210eda0810947c6e6611704d119ac575d79cfd10c88b81fe3db9251",
                    "Gateway": "<IP_ADDRESS>",
                    "IPAddress": "<IP_ADDRESS>",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:10:26:02",
                    "DriverOpts": null
                }
            }
        }
    }
]

Okay, I have found the mistake. When I write "25:25" instead of 25:25 (without the quotation marks) in the configuration file called docker-compose.yml it is working. With all other ports it works with and without quotation marks.

@timotion83 I did not see that one, I used the example files that are in the readme and they are with quotes.
Please don't forget to close the ticket and your welcome.

Yes, the example files are correct. The mistake was on my side. It was only confusing that it worked with all the other ports. Thanks a lot.


JSON schema creation on llama cpp failing

Describe the issue as clearly as possible:
I'm trying to generate using a JSON schema on llama.cpp. This throws lots of errors and creates wrong output.
Steps/code to reproduce the bug:
import outlines
import json
import os
import sys

def raw_outlines_test():
    """
    Just a test

    """
    schema = """{"properties": {"name": {"title": "Name", "type": "string",
    "maxLength": 100}},
    "required": ["name"], "title": "person", "type": "object"}
    """
    import sys
    json.loads(schema)
    sys.stderr.write("XXXXXXXXXXX  JSON schema successfully created  XXXXXXXXXXXXXX\n")
    llm = outlines.models.llamacpp(
        os.path.join(
            os.environ["TRANSFORMERS_CACHE"],
            "Nous-Capybara-34B-GGUF/nous-capybara-34b.Q4_0.gguf"),
        model_kwargs = {"n_gpu_layers": 15000, "n_ctx": 5 * 1024})
    sys.stderr.write("XXXXXXXXXX Model loaded  XXXXXXXXXXXXXXXXXXXXXX\n")
    generator = outlines.generate.json(llm, schema)
    sys.stderr.write("XXXXXXXXXXXX Generator created XXXXXXXXXXXXXXX\n")
    text = generator("What is the name of Michael Jackson?")
    sys.stderr.write("XXXXXXXXXXXXX Result: XXXXXXXXXXXXXXXX\n")
    sys.stderr.write(str(text))

Expected result:
{
	"name": "Michael Jackson"
}

Error message:
XXXXXXXXXXX  JSON schema successfully created  XXXXXXXXXXXXXX
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 CUDA devices:
  Device 0: NVIDIA A100-SXM4-40GB, compute capability 8.0, VMM: yes
llama_model_loader: loaded meta data with 22 key-value pairs and 543 tensors from /misc/cache/huggingface/models/Nous-Capybara-34B-GGUF/nous-capybara-34b.Q4_0.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.name str              = nousresearch_nous-capybara-34b
llama_model_loader: - kv   2:                       llama.context_length u32              = 200000
llama_model_loader: - kv   3:                     llama.embedding_length u32              = 7168
llama_model_loader: - kv   4:                          llama.block_count u32              = 60
llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 20480
llama_model_loader: - kv   6:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv   7:                 llama.attention.head_count u32              = 56
llama_model_loader: - kv   8:              llama.attention.head_count_kv u32              = 8
llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
llama_model_loader: - kv  10:                       llama.rope.freq_base f32              = 5000000.000000
llama_model_loader: - kv  11:                          general.file_type u32              = 2
llama_model_loader: - kv  12:                       tokenizer.ggml.model str              = llama
llama_model_loader: - kv  13:                      tokenizer.ggml.tokens arr[str,64000]   = ["<unk>", "<|startoftext|>", "<|endof...
llama_model_loader: - kv  14:                      tokenizer.ggml.scores arr[f32,64000]   = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr[i32,64000]   = [2, 3, 3, 3, 3, 3, 1, 1, 1, 3, 3, 3, ...
llama_model_loader: - kv  16:                tokenizer.ggml.bos_token_id u32              = 1
llama_model_loader: - kv  17:                tokenizer.ggml.eos_token_id u32              = 2
llama_model_loader: - kv  18:            tokenizer.ggml.padding_token_id u32              = 0
llama_model_loader: - kv  19:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  20:               tokenizer.ggml.add_eos_token bool             = false
llama_model_loader: - kv  21:               general.quantization_version u32              = 2
llama_model_loader: - type  f32:  121 tensors
llama_model_loader: - type q4_0:  421 tensors
llama_model_loader: - type q6_K:    1 tensors
llm_load_vocab: mismatch in special tokens definition ( 498/64000 vs 267/64000 ).
llm_load_print_meta: format           = GGUF V3 (latest)
llm_load_print_meta: arch             = llama
llm_load_print_meta: vocab type       = SPM
llm_load_print_meta: n_vocab          = 64000
llm_load_print_meta: n_merges         = 0
llm_load_print_meta: n_ctx_train      = 200000
llm_load_print_meta: n_embd           = 7168
llm_load_print_meta: n_head           = 56
llm_load_print_meta: n_head_kv        = 8
llm_load_print_meta: n_layer          = 60
llm_load_print_meta: n_rot            = 128
llm_load_print_meta: n_embd_head_k    = 128
llm_load_print_meta: n_embd_head_v    = 128
llm_load_print_meta: n_gqa            = 7
llm_load_print_meta: n_embd_k_gqa     = 1024
llm_load_print_meta: n_embd_v_gqa     = 1024
llm_load_print_meta: f_norm_eps       = 0.0e+00
llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
llm_load_print_meta: f_clamp_kqv      = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: n_ff             = 20480
llm_load_print_meta: n_expert         = 0
llm_load_print_meta: n_expert_used    = 0
llm_load_print_meta: rope scaling     = linear
llm_load_print_meta: freq_base_train  = 5000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx  = 200000
llm_load_print_meta: rope_finetuned   = unknown
llm_load_print_meta: model type       = 30B
llm_load_print_meta: model ftype      = Q4_0
llm_load_print_meta: model params     = 34.39 B
llm_load_print_meta: model size       = 18.13 GiB (4.53 BPW)
llm_load_print_meta: general.name     = nousresearch_nous-capybara-34b
llm_load_print_meta: BOS token        = 1 '<|startoftext|>'
llm_load_print_meta: EOS token        = 2 '<|endoftext|>'
llm_load_print_meta: UNK token        = 0 '<unk>'
llm_load_print_meta: PAD token        = 0 '<unk>'
llm_load_print_meta: LF token         = 315 '<0x0A>'
llm_load_tensors: ggml ctx size =    0.41 MiB
llm_load_tensors: offloading 60 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 61/61 layers to GPU
llm_load_tensors:        CPU buffer size =   246.09 MiB
llm_load_tensors:      CUDA0 buffer size = 18317.20 MiB
...................................................................................................
Failed to convert token (b'\x80'): 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
Failed to convert token (b'\x81'): 'utf-8' codec can't decode byte 0x81 in position 0: invalid start byte
Failed to convert token (b'\x82'): 'utf-8' codec can't decode byte 0x82 in position 0: invalid start byte
Failed to convert token (b'\x83'): 'utf-8' codec can't decode byte 0x83 in position 0: invalid start byte
Failed to convert token (b'\x84'): 'utf-8' codec can't decode byte 0x84 in position 0: invalid start byte
Failed to convert token (b'\x85'): 'utf-8' codec can't decode byte 0x85 in position 0: invalid start byte
Failed to convert token (b'\x86'): 'utf-8' codec can't decode byte 0x86 in position 0: invalid start byte
Failed to convert token (b'\x87'): 'utf-8' codec can't decode byte 0x87 in position 0: invalid start byte
Failed to convert token (b'\x88'): 'utf-8' codec can't decode byte 0x88 in position 0: invalid start byte
Failed to convert token (b'\x89'): 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
Failed to convert token (b'\x8a'): 'utf-8' codec can't decode byte 0x8a in position 0: invalid start byte
Failed to convert token (b'\x8b'): 'utf-8' codec can't decode byte 0x8b in position 0: invalid start byte
Failed to convert token (b'\x8c'): 'utf-8' codec can't decode byte 0x8c in position 0: invalid start byte
Failed to convert token (b'\x8d'): 'utf-8' codec can't decode byte 0x8d in position 0: invalid start byte
Failed to convert token (b'\x8e'): 'utf-8' codec can't decode byte 0x8e in position 0: invalid start byte
Failed to convert token (b'\x8f'): 'utf-8' codec can't decode byte 0x8f in position 0: invalid start byte
Failed to convert token (b'\x90'): 'utf-8' codec can't decode byte 0x90 in position 0: invalid start byte
Failed to convert token (b'\x91'): 'utf-8' codec can't decode byte 0x91 in position 0: invalid start byte
Failed to convert token (b'\x92'): 'utf-8' codec can't decode byte 0x92 in position 0: invalid start byte
Failed to convert token (b'\x93'): 'utf-8' codec can't decode byte 0x93 in position 0: invalid start byte
Failed to convert token (b'\x94'): 'utf-8' codec can't decode byte 0x94 in position 0: invalid start byte
Failed to convert token (b'\x95'): 'utf-8' codec can't decode byte 0x95 in position 0: invalid start byte
Failed to convert token (b'\x96'): 'utf-8' codec can't decode byte 0x96 in position 0: invalid start byte
Failed to convert token (b'\x97'): 'utf-8' codec can't decode byte 0x97 in position 0: invalid start byte
Failed to convert token (b'\x98'): 'utf-8' codec can't decode byte 0x98 in position 0: invalid start byte
Failed to convert token (b'\x99'): 'utf-8' codec can't decode byte 0x99 in position 0: invalid start byte
Failed to convert token (b'\x9a'): 'utf-8' codec can't decode byte 0x9a in position 0: invalid start byte
Failed to convert token (b'\x9b'): 'utf-8' codec can't decode byte 0x9b in position 0: invalid start byte
Failed to convert token (b'\x9c'): 'utf-8' codec can't decode byte 0x9c in position 0: invalid start byte
Failed to convert token (b'\x9d'): 'utf-8' codec can't decode byte 0x9d in position 0: invalid start byte
Failed to convert token (b'\x9e'): 'utf-8' codec can't decode byte 0x9e in position 0: invalid start byte
Failed to convert token (b'\x9f'): 'utf-8' codec can't decode byte 0x9f in position 0: invalid start byte
Failed to convert token (b'\xa0'): 'utf-8' codec can't decode byte 0xa0 in position 0: invalid start byte
Failed to convert token (b'\xa1'): 'utf-8' codec can't decode byte 0xa1 in position 0: invalid start byte
Failed to convert token (b'\xa2'): 'utf-8' codec can't decode byte 0xa2 in position 0: invalid start byte
Failed to convert token (b'\xa3'): 'utf-8' codec can't decode byte 0xa3 in position 0: invalid start byte
Failed to convert token (b'\xa4'): 'utf-8' codec can't decode byte 0xa4 in position 0: invalid start byte
Failed to convert token (b'\xa5'): 'utf-8' codec can't decode byte 0xa5 in position 0: invalid start byte
Failed to convert token (b'\xa6'): 'utf-8' codec can't decode byte 0xa6 in position 0: invalid start byte
Failed to convert token (b'\xa7'): 'utf-8' codec can't decode byte 0xa7 in position 0: invalid start byte
Failed to convert token (b'\xa8'): 'utf-8' codec can't decode byte 0xa8 in position 0: invalid start byte
Failed to convert token (b'\xa9'): 'utf-8' codec can't decode byte 0xa9 in position 0: invalid start byte
Failed to convert token (b'\xaa'): 'utf-8' codec can't decode byte 0xaa in position 0: invalid start byte
Failed to convert token (b'\xab'): 'utf-8' codec can't decode byte 0xab in position 0: invalid start byte
Failed to convert token (b'\xac'): 'utf-8' codec can't decode byte 0xac in position 0: invalid start byte
Failed to convert token (b'\xad'): 'utf-8' codec can't decode byte 0xad in position 0: invalid start byte
Failed to convert token (b'\xae'): 'utf-8' codec can't decode byte 0xae in position 0: invalid start byte
Failed to convert token (b'\xaf'): 'utf-8' codec can't decode byte 0xaf in position 0: invalid start byte
Failed to convert token (b'\xb0'): 'utf-8' codec can't decode byte 0xb0 in position 0: invalid start byte
Failed to convert token (b'\xb1'): 'utf-8' codec can't decode byte 0xb1 in position 0: invalid start byte
Failed to convert token (b'\xb2'): 'utf-8' codec can't decode byte 0xb2 in position 0: invalid start byte
Failed to convert token (b'\xb3'): 'utf-8' codec can't decode byte 0xb3 in position 0: invalid start byte
Failed to convert token (b'\xb4'): 'utf-8' codec can't decode byte 0xb4 in position 0: invalid start byte
Failed to convert token (b'\xb5'): 'utf-8' codec can't decode byte 0xb5 in position 0: invalid start byte
Failed to convert token (b'\xb6'): 'utf-8' codec can't decode byte 0xb6 in position 0: invalid start byte
Failed to convert token (b'\xb7'): 'utf-8' codec can't decode byte 0xb7 in position 0: invalid start byte
Failed to convert token (b'\xb8'): 'utf-8' codec can't decode byte 0xb8 in position 0: invalid start byte
Failed to convert token (b'\xb9'): 'utf-8' codec can't decode byte 0xb9 in position 0: invalid start byte
Failed to convert token (b'\xba'): 'utf-8' codec can't decode byte 0xba in position 0: invalid start byte
Failed to convert token (b'\xbb'): 'utf-8' codec can't decode byte 0xbb in position 0: invalid start byte
Failed to convert token (b'\xbc'): 'utf-8' codec can't decode byte 0xbc in position 0: invalid start byte
Failed to convert token (b'\xbd'): 'utf-8' codec can't decode byte 0xbd in position 0: invalid start byte
Failed to convert token (b'\xbe'): 'utf-8' codec can't decode byte 0xbe in position 0: invalid start byte
Failed to convert token (b'\xbf'): 'utf-8' codec can't decode byte 0xbf in position 0: invalid start byte
Failed to convert token (b'\xc0'): 'utf-8' codec can't decode byte 0xc0 in position 0: invalid start byte
Failed to convert token (b'\xc1'): 'utf-8' codec can't decode byte 0xc1 in position 0: invalid start byte
Failed to convert token (b'\xc2'): 'utf-8' codec can't decode byte 0xc2 in position 0: unexpected end of data
Failed to convert token (b'\xc3'): 'utf-8' codec can't decode byte 0xc3 in position 0: unexpected end of data
Failed to convert token (b'\xc4'): 'utf-8' codec can't decode byte 0xc4 in position 0: unexpected end of data
Failed to convert token (b'\xc5'): 'utf-8' codec can't decode byte 0xc5 in position 0: unexpected end of data
Failed to convert token (b'\xc6'): 'utf-8' codec can't decode byte 0xc6 in position 0: unexpected end of data
Failed to convert token (b'\xc7'): 'utf-8' codec can't decode byte 0xc7 in position 0: unexpected end of data
Failed to convert token (b'\xc8'): 'utf-8' codec can't decode byte 0xc8 in position 0: unexpected end of data
Failed to convert token (b'\xc9'): 'utf-8' codec can't decode byte 0xc9 in position 0: unexpected end of data
Failed to convert token (b'\xca'): 'utf-8' codec can't decode byte 0xca in position 0: unexpected end of data
Failed to convert token (b'\xcb'): 'utf-8' codec can't decode byte 0xcb in position 0: unexpected end of data
llama_new_context_with_model: n_ctx      = 5120
llama_new_context_with_model: freq_base  = 5000000.0
llama_new_context_with_model: freq_scale = 1
llama_kv_cache_init:      CUDA0 KV buffer size =  1200.00 MiB
llama_new_context_with_model: KV self size  = 1200.00 MiB, K (f16):  600.00 MiB, V (f16):  600.00 MiB
llama_new_context_with_model:  CUDA_Host input buffer size   =    24.02 MiB
llama_new_context_with_model:      CUDA0 compute buffer size =   673.20 MiB
llama_new_context_with_model:  CUDA_Host compute buffer size =    15.40 MiB
llama_new_context_with_model: graph splits (measure): 3
XXXXXXXXXX Model loaded  XXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXX Generator created XXXXXXXXXXXXXXX
XXXXXXXXXXXXX Result: XXXXXXXXXXXXXXXX
Failed to convert token (b'\xcc'): 'utf-8' codec can't decode byte 0xcc in position 0: unexpected end of data
Failed to convert token (b'\xcd'): 'utf-8' codec can't decode byte 0xcd in position 0: unexpected end of data
Failed to convert token (b'\xce'): 'utf-8' codec can't decode byte 0xce in position 0: unexpected end of data
Failed to convert token (b'\xcf'): 'utf-8' codec can't decode byte 0xcf in position 0: unexpected end of data
Failed to convert token (b'\xd0'): 'utf-8' codec can't decode byte 0xd0 in position 0: unexpected end of data
Failed to convert token (b'\xd1'): 'utf-8' codec can't decode byte 0xd1 in position 0: unexpected end of data
Failed to convert token (b'\xd2'): 'utf-8' codec can't decode byte 0xd2 in position 0: unexpected end of data
Failed to convert token (b'\xd3'): 'utf-8' codec can't decode byte 0xd3 in position 0: unexpected end of data
Failed to convert token (b'\xd4'): 'utf-8' codec can't decode byte 0xd4 in position 0: unexpected end of data
Failed to convert token (b'\xd5'): 'utf-8' codec can't decode byte 0xd5 in position 0: unexpected end of data
Failed to convert token (b'\xd6'): 'utf-8' codec can't decode byte 0xd6 in position 0: unexpected end of data
Failed to convert token (b'\xd7'): 'utf-8' codec can't decode byte 0xd7 in position 0: unexpected end of data
Failed to convert token (b'\xd8'): 'utf-8' codec can't decode byte 0xd8 in position 0: unexpected end of data
Failed to convert token (b'\xd9'): 'utf-8' codec can't decode byte 0xd9 in position 0: unexpected end of data
Failed to convert token (b'\xda'): 'utf-8' codec can't decode byte 0xda in position 0: unexpected end of data
Failed to convert token (b'\xdb'): 'utf-8' codec can't decode byte 0xdb in position 0: unexpected end of data
Failed to convert token (b'\xdc'): 'utf-8' codec can't decode byte 0xdc in position 0: unexpected end of data
Failed to convert token (b'\xdd'): 'utf-8' codec can't decode byte 0xdd in position 0: unexpected end of data
Failed to convert token (b'\xde'): 'utf-8' codec can't decode byte 0xde in position 0: unexpected end of data
Failed to convert token (b'\xdf'): 'utf-8' codec can't decode byte 0xdf in position 0: unexpected end of data
Failed to convert token (b'\xe0'): 'utf-8' codec can't decode byte 0xe0 in position 0: unexpected end of data
Failed to convert token (b'\xe1'): 'utf-8' codec can't decode byte 0xe1 in position 0: unexpected end of data
Failed to convert token (b'\xe2'): 'utf-8' codec can't decode byte 0xe2 in position 0: unexpected end of data
Failed to convert token (b'\xe3'): 'utf-8' codec can't decode byte 0xe3 in position 0: unexpected end of data
Failed to convert token (b'\xe4'): 'utf-8' codec can't decode byte 0xe4 in position 0: unexpected end of data
Failed to convert token (b'\xe5'): 'utf-8' codec can't decode byte 0xe5 in position 0: unexpected end of data
Failed to convert token (b'\xe6'): 'utf-8' codec can't decode byte 0xe6 in position 0: unexpected end of data
Failed to convert token (b'\xe7'): 'utf-8' codec can't decode byte 0xe7 in position 0: unexpected end of data
Failed to convert token (b'\xe8'): 'utf-8' codec can't decode byte 0xe8 in position 0: unexpected end of data
Failed to convert token (b'\xe9'): 'utf-8' codec can't decode byte 0xe9 in position 0: unexpected end of data
Failed to convert token (b'\xea'): 'utf-8' codec can't decode byte 0xea in position 0: unexpected end of data
Failed to convert token (b'\xeb'): 'utf-8' codec can't decode byte 0xeb in position 0: unexpected end of data
Failed to convert token (b'\xec'): 'utf-8' codec can't decode byte 0xec in position 0: unexpected end of data
Failed to convert token (b'\xed'): 'utf-8' codec can't decode byte 0xed in position 0: unexpected end of data
Failed to convert token (b'\xee'): 'utf-8' codec can't decode byte 0xee in position 0: unexpected end of data
Failed to convert token (b'\xef'): 'utf-8' codec can't decode byte 0xef in position 0: unexpected end of data
Failed to convert token (b'\xf0'): 'utf-8' codec can't decode byte 0xf0 in position 0: unexpected end of data
Failed to convert token (b'\xf1'): 'utf-8' codec can't decode byte 0xf1 in position 0: unexpected end of data
Failed to convert token (b'\xf2'): 'utf-8' codec can't decode byte 0xf2 in position 0: unexpected end of data
Failed to convert token (b'\xf3'): 'utf-8' codec can't decode byte 0xf3 in position 0: unexpected end of data
Failed to convert token (b'\xf4'): 'utf-8' codec can't decode byte 0xf4 in position 0: unexpected end of data
Failed to convert token (b'\xf5'): 'utf-8' codec can't decode byte 0xf5 in position 0: invalid start byte
Failed to convert token (b'\xf6'): 'utf-8' codec can't decode byte 0xf6 in position 0: invalid start byte
Failed to convert token (b'\xf7'): 'utf-8' codec can't decode byte 0xf7 in position 0: invalid start byte
Failed to convert token (b'\xf8'): 'utf-8' codec can't decode byte 0xf8 in position 0: invalid start byte
Failed to convert token (b'\xf9'): 'utf-8' codec can't decode byte 0xf9 in position 0: invalid start byte
Failed to convert token (b'\xfa'): 'utf-8' codec can't decode byte 0xfa in position 0: invalid start byte
Failed to convert token (b'\xfb'): 'utf-8' codec can't decode byte 0xfb in position 0: invalid start byte
Failed to convert token (b'\xfc'): 'utf-8' codec can't decode byte 0xfc in position 0: invalid start byte
Failed to convert token (b'\xfd'): 'utf-8' codec can't decode byte 0xfd in position 0: invalid start byte
Failed to convert token (b'\xfe'): 'utf-8' codec can't decode byte 0xfe in position 0: invalid start byte
Failed to convert token (b'\xff'): 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
{'name': "      [TYPE                     : String     <'aecollertlqpangstrom'      )-> int};</ pb>q_ -> reg->"}

Outlines/Python version information:
Version information

```
0.0.25
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
</details>

### Context for the issue:

Outlines is currently not usable for me as I'm relying on a JSON schema functionality.

It's 0.2.36, so I'll upgrade and retry,
greetings!

The upgrade didn't change much. The error messages seem unchanged; the returned value is, most of the time, {'name': '               '}, but scarier returns of strange memory contents ({'name': '}</s><commit_msg>Update Italian translation<commit_after>{</<s>{name> configuration values frompt-</'}) happen as well. I once even saw the string "passwd" in the returned data...

Thanks for trying, sorry it's not still working as expected. Will investigate.

So you can reproduce the error? That gives me at least some peace of mind
as it's not related to my setup...
Andrew Lapp @.***> schrieb am Do., 8. Feb. 2024, 04:29:

Thanks for trying, sorry it's not still working as expected. Will
investigate.
—
Reply to this email directly, view it on GitHub
https://github.com/outlines-dev/outlines/issues/611#issuecomment-1933306973,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACO5DS54SLJMHDPBX33FIB3YSRBBBAVCNFSM6AAAAABC2JJ2JCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZTGMYDMOJXGM
.
You are receiving this because you authored the thread.Message ID:
@.***>

Dear all,
It is my first message ever on GitHub. Pardon me if I am doing something wrong.
Yesterday, I installed the last version of outlines and python-llama-cpp and I have a similar series of error messages "failed to convert token", whatever the prompt or the gguf model loaded. I am using CPU only.

There is a PR open that should solve most issues with llama.cpp (https://github.com/outlines-dev/outlines/pull/556), I will review it as quickly as I can

@motaatmo my test suggests your issue is resolved by https://github.com/outlines-dev/outlines/pull/556
You may try it early via pip install git+https://github.com/dtiarks/outlines@llama-logits-processor && pip install llama-cpp-python==0.2.37

Yes, it works! No more errors, however one minor thing: the returned JSON is using single rather than double quotes, which is, as far as I know, not JSON-compliant

Yes, it works! No more errors, however one minor thing: the returned JSON is using single rather than double quotes, which is, as far as I know, not JSON-compliant

This shouldn't be possible with Outlines and is a bug. Could you share the script you're running in a new issue?


List.Zipper.Empty

Hey there! Thanks for maintaining this library, I've been using it a ton! Recently I ran into a case where I basically needed a zipper in a maybe, basically a list that's either empty or has an element with a focus:
type EmptyZipper a
    = Empty
    | NotEmpty (Zipper a)

It implements the same API as the Zipper a, apart from a Maybe a occurring in a different place now and then.
I'm wondering if this is useful beyond my particular use case, and if so whether this is something you'd like to be part of your library. If yes on both counts I'm happy to make submit the code I have as a PR for review. And if not that's of course also fine!

It implements the same API as the Zipper a, apart from a Maybe a occurring in a different place now and then.

Am I correct in assuming that it mostly just wrappers the functions in a Maybe.map?

I don't use a Maybe anymore, but basically that's what it comes down to. If you're interested, I put the module in a Gist here: https://gist.github.com/jwoudenberg/2db3d5c7412ca2195efe72f54940a234

I think I understand! It lets you construct a Zipper for an empty List. Other than that, it's completely the same as Zipper. Is that correct?

Yep, that's pretty much it!

I'm going to chew on this for a little while. I have the feeling that #19 will be a major change, and I want your request (as this is something I feel many people would like) to nicely integrate with all these changes.

Thanks for the gist, @jwoudenberg .
For anyone else that finds themselves needing a non-empty Zipper, consider that maybe what you need is a List or a Zipper, and simply convert between the two.
I went down the non-empty Zipper path, but found that in actuality, if my list could be empty, then it didn't need to be a zipper at all (since the point of a zipper is "always one selected", which is incompatible with "none exist").
Might not apply to all, but thought I'd include this here, in case anyone else comes across this and finds it useful.
Thanks for a great package :+1:


Events with start time show as full day event

Is there anyway to get an event that only has a start time to not show up as an all day event?

Obvious question.
All day event - Starts from 00:00AM and ends in 24:00PM(of any day). So, start & end time is meaningless.
Which case did you mean? give me the example.

The event has a start time but end time. See attached picture for events on
10/6 & 10/7...
Here is the calendar link to test:
webcal://www.cheektowagawarriors.net/ical_feed?tags=4108079
Nick
On Fri, Oct 5, 2018, 3:33 AM Seongnoh Sean Yi<EMAIL_ADDRESS>wrote:

Obvious question.
All day event - Starts from 00:00AM and ends in 24:00PM(of any day). So,
start & end time is meaningless.
Which case did you mean? give me the example.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/eouia/MMM-CalendarExt/issues/44#issuecomment-427272980,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Anbl4ouPkLIx0d5pNGpvq_XEWB-13kx2ks5uhwssgaJpZM4XJcPu
.

No I am ok with the full day event showing on both days.
There is a second event on 10/6 from that calendar "Batavia at Peewee Minor Red" with a start time of 4:00pm, but no end time. In month view which is my primary view that event shows as an all day event also.

I've fixed. try it.

I'll try shortly, thanks!

I'll close this.


[WIP] Support for null values returned from lists

Just wanted to make sure we all liked this path (per #65). Pretty straightforward, but I do want to add some more tests and clean up.
Basically, if an item in the array of objects is null, the store is populated like this:
{ abcd:
   { id: 'abcd',
     nestedArray: [ 'abcd.nestedArray.0', 'abcd.nestedArray.1' ] },
  'abcd.nestedArray.0': null,
  'abcd.nestedArray.1':
   { stringField: 'This is a string also!',
     numberField: 7,
     nullField: null } }

Thoughts?

Hmmm... what are the tradeoffs vs. putting null in the nestedArray in the store directly?
I think this will be the first time we have key: null in the store, since nested objects are not handled in this way AFAIK?

Yeah, confirmed that nested objects are handled differently: https://github.com/apollostack/apollo-client/blob/e373618bf63960251805d891544e6602e605a5af/test/writeToStore.ts#L518

@stubalio I think the expectation is the objects in a normalized array would be stored in the same manner. I like having the null normalized fwiw

@stubailo are you suggesting this?
{ abcd:
   { id: 'abcd',
     nestedArray: [null, 'abcd.nestedArray.1' ] },
  'abcd.nestedArray.1':
   { stringField: 'This is a string also!',
     numberField: 7,
     nullField: null } }

I like the idea of the items ending up in the same place whether they are null or not, but I don't necessarily have a problem with leaving null in the array.

Actually I just realized what I don't like about this -- imagine that the items in the array have IDs:
{ abcd:
   { id: 'abcd',
     nestedArray: [null, 'theObjectId' ] },
  'theObjectId':
   {
     id: 'theObjectId',
     stringField: 'This is a string also!',
     numberField: 7,
     nullField: null } }

If we denormalized the null, what would we use for the key?

Dang, that's a good point, without generating a custom id (bad idea if other ids are present) this wouldn't work out

Agreed. And that actually solves the next test I was going to add haha. I'll run with keeping it in the array.

closing in favor of #70


tab completion not working in jupyterlite when using xeus-python kernel

To reproduce:

go to https://xeus-python-kernel.readthedocs.io/en/latest/
type pr<TAB> no completions is shown

Expected behaviour: completions are shown showing print for example
Observed behaviour: no completion is shown.
I originally observed the behaviour when using jupyterlite build --XeusPython.packages locally
I tried both in Firefox and Chromium, same behaviour.
Pyolite tab-completion seems to work fine e.g. on https://numpy.org/
Output from the browser console:
ERROR: received bad message: NotImplementedError: Python version 3.10 is currently not supported. xpython_wasm.js:9:53168
<empty string> xpython_wasm.js:9:53168
At: xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/parso/grammar.py(264): load_grammar xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/api/environment.py(37): get_grammar xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/cache.py(112): wrapper xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/inference/__init__.py(91): __init__ xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/api/__init__.py(119): __init__ xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/jedi/api/__init__.py(729): __init__ xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/IPython/core/completer.py(1397): _jedi_matches xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/IPython/core/completer.py(2054): _complete xpython_wasm.js:9:53168
  /tmp/xeus-python-kernel/envs/xeus-python-kernel/lib/python3.10/site-packages/IPython/core/completer.py(1886): _completions xpython_wasm.js:9:53168
<empty string> xpython_wasm.js:9:53168
Message type: complete_request xpython_wasm.js:9:53168

Here is a screenshot in case that helps

Thank you for opening an issue!
We should probably check if we're using the latest version of parso/jedi.

Inside the Jupyterlite from https://xeus-python-kernel.readthedocs.io/en/latest/,

import jedi; print(f"{jedi.__version__=}")
import parso; print(f"{parso.__version__=}")

jedi.__version__='0.18.1'
parso.__version__='0.8.3'

This seems like the latest versions https://pypi.org/project/jedi/#history and https://pypi.org/project/parso/#history

For reference this still seems to be the case with the latest versions of jupyterlite-xeus-python and jedi / parso:

Just checked with the latest jedi 0.19 and there is still the same issue:

Using the https://xeus-python-kernel.readthedocs.io deployment I see completion working:

Although it's true that pr<tab> does not show print as an option for some reason:

This is what I get with the following code snippet and pressing TAB:
import sys
sys.

https://github.com/jupyterlite/xeus-python-kernel/assets/591645/a9990b7c-1fdf-4b56-b638-1503cbd1d52a

Yep, that seems to do the trick:

We can probably disable jedi in xeus-python by default. I would not be against this.

Also what does ipykernel do to provide "nicer" completions out of the box compared to xeus python?
ipykernel

xeus-python

We're probably missing some metadata in the completion answer.

Indeed, when using jedi, ipykernel fills up some metadata:
https://github.com/ipython/ipykernel/blob/c8517461f434a72fb4e30fc03b5949aca4c5af52/ipykernel/ipkernel.py#L540C26-L540C48

And JupyterLab uses this information here https://github.com/jupyterlab/jupyterlab/blob/bddc378b72e8ee78be8b6e2ffb2b527a75f0471a/packages/completer/src/default/kernelprovider.ts#L61
We may want to explore implementing the same when not using Jedi. It might be possible, depending on what the IPython's Interactive Shell Completer gives us as information.


DarkSectionPanel Anchor Doesn't Work Properly?

After using the designer to set controls (like DarkSectionPanel) anchor properties that are mostly at the edge of my DarkForm, such as doing Left, Top, Bottom so it's stuck to the left side of the form. Running the app in the normal window state and then maximising doesn't keep the anchoring so that they're stretched across the side of the forms. If you would like to me to include some screenshots to show what I mean. Thanks.

Sorry, I'm not entirely sure what you're asking. As you've closed the issue, I'm assuming you've managed to fix it?

That's my bad I was having a moment. I forgot there was a container for my controls forgot to  and set it's anchor properties.


[improve] update docker-compose license and fix compose config yml

What's changed?

update docker-compose license
fix compose config yml
update template yml

Checklist

[ ]  I have read the Contributing Guide
[ ]  I have written the necessary doc or comment.
[ ]  I have added the necessary unit tests and all cases have passed.

Add or update API

[ ] I have added the necessary e2e tests and all cases have passed.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 31.82%. Comparing base (efa2dc3) to head (d15ec41).
Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2778      +/-   ##
============================================
+ Coverage     31.80%   31.82%   +0.01%
  Complexity     1706     1706
============================================
  Files           410      410
  Lines         16540    16540
  Branches       2138     2138
============================================
+ Hits           5261     5264       +3
+ Misses        10641    10638       -3
  Partials        638      638

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


400 Bad Request while posting to InfluxDB

Hi,
For a project we are trying to monitor a status protocol sent on local network over TCP and post these status messages over HTTPS to a InfluxDB instance on AWS at https://www.influxdata.com/products/influxdb-cloud/.
Posting a string to a test instance over LAN to a IP works fine, but whenever we try to post to the AWS subdomain we are greeted with this response:
HTTP/1.1 400 Bad Request
Server: awselb/2.0
Date: Mon, 27 Sep 2021 18:13:02 GMT
Content-Type: text/html
Content-Length: 122
Connection: close

400 Bad Request

400 Bad Request

Since we verified the headers and body of the post are correct when we post locally, and posting to AWS works when using Postman, we are at a loss at the moment. Unfortunately I have no information about the configuration of the server we try to POST to.
As a test we set-up the WebClientRepeatingTLS example using our own POST headers and body.
#include <NativeEthernet.h>

uint8_t mac[6];
void teensyMAC(uint8_t *mac) {
  for (uint8_t by = 0; by < 2; by++) mac[by] = (HW_OCOTP_MAC1 >> ((1 - by) * 8)) & 0xFF;
  for (uint8_t by = 0; by < 4; by++) mac[by + 2] = (HW_OCOTP_MAC0 >> ((3 - by) * 8)) & 0xFF;
  Serial.printf("MAC: %02x:%02x:%02x:%02x:%02x:%02x\n", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
}

// initialize the library instance:
EthernetClient client;
const int port = 443;

char server[] = "influxdata.com";

//IPAddress server(192, 168, 1, 246);

String outputMSG = "measurementName,tagKey=tagValue fieldKey=1i";
unsigned long lastConnectionTime = 0;           // last time you connected to the server, in milliseconds
const unsigned long postingInterval = 5 * 1000; // delay between updates, in milliseconds

void setup() {
  teensyMAC(mac);
  Serial.begin(9600);
  while (!Serial) {
    ;
  }

  Serial.println("Initialize Ethernet with DHCP:");
  if (Ethernet.begin(mac) == 0) {
    Serial.println("Failed to configure Ethernet using DHCP");
    if (Ethernet.hardwareStatus() == EthernetNoHardware) {
      Serial.println("Ethernet shield was not found.  Sorry, can't run without hardware. :(");
      while (true) {
        delay(1);
      }
    }
    if (Ethernet.linkStatus() == LinkOFF) {
      Serial.println("Ethernet cable is not connected.");
    }
  } else {
    Serial.print("  DHCP assigned IP ");
    Serial.println(Ethernet.localIP());
  }
  delay(1000);
}

void loop() {
  if (client.available()) {
    char c = client.read();
    Serial.write(c);
  }
  if (millis() - lastConnectionTime > postingInterval) {
    httpRequest();
  }

}

void httpRequest() {
  client.stop();
  Serial.print("Content-Length: ");
  Serial.println(outputMSG.length());
  Serial.println("");

  if (client.connect(server, port, true)) {
    Serial.println("connecting to ");
    Serial.print(server);
    Serial.print(":");
    Serial.println(port);

        client.println("POST /api/v2/write?org=ORG_HERE&bucket=BUCKET_HERE&precision=s HTTP/1.1");
        client.println("Host: https://eu-central-1-1.aws.cloud2.influxdata.com");
        client.println("Authorization: Token <<TOKEN HERE>>");
        client.println("Content-Type: text/plain");
        client.print("Content-Length: ");
        client.println(outputMSG.length());
        client.println("Accept-Encoding: gzip, deflate, br");
        client.println("Connection: keep-alive");
        client.println();
        client.println(outputMSG);
        client.println();

    lastConnectionTime = millis();
  } else {
    Serial.println("connection failed");
  }

Could anyone help us understand why the post to aws is not going through?
Cheers,
Boy

I went with QNEthernet and EthernetWebServer_SSL as per vjmuzik's suggestion in the link below, now it works.
Pjrc forum discussion


emacs tab completion popup does not display absolute value

I don't know if there is an easy way to fix this. It would be o.k. to use "abs t" instead of "|t|".

@avigad, I'm on it.

@leodemoura, we need to escape the symbol | which we use as a delimiter in a packet between lean-server and an emacs client. Here is the text we get for the above example:
int.abs.by_cases|?P ?a → ?P (- ?a) → ?P | ?a |
                                        ^    ^

For this particular case (type information in auto-complete), I can avoid the problem by only splitting a line at the first occurrence of | and accept the rest of the line as a type.
Maybe an easier solution to implement is to use a multi-char delimiter to reduce the possibility of having such a conflict?

@avigad and @soonhokong
This would be great. If we stop using |a| for abs a, then | can be a separator (like ,).
Separators are tokens with binding power 0.
We can improve the syntax in many places. Right now, we rely too much on ,.
Here are some examples:
1- Nice subtypes { x : A | P x }. We have this notation, but since | does not have binding power 0, we sometimes have to use parenthesis around the type.
2- ML-like notation for recursive equations and inductive datatypes
inductive nat :=
| zero : nat
| succ : nat -> nat

definition add : nat -> nat -> nat
| add 0        b := b
| add (succ a) b := succ (add a b)

@soonhokong
Thanks for finding the problem.
I think we should escape the character | as you suggested.
I can do that in the Lean server.
That being said, I still think it would be nice to stop using |a| for absolute value.

Here is another example.
Current syntax:
definition nth [h : inhabited T] : list T → nat → T,
nth nil      n     := arbitrary T,
nth (a :: l) 0     := a,
nth (a :: l) (n+1) := nth l n

New syntax:
definition nth [h : inhabited T] : list T → nat → T
| nth nil      n     := arbitrary T
| nth (a :: l) 0     := a
| nth (a :: l) (n+1) := nth l n

I think we should escape the character | as you suggested. I can do that in the Lean server.

Thank you. I'll also work on emacs-side for that change.
Regarding the notation for absolute value, for your information, we have some unicode characters such as

‖ (U+2016, double vertical line)
| (U+007C, vertical line)

and people can still define their own notations for abs and norm using those unicode characters. Note that we already have the entries for them in lean mode's input-method -- \| for | and \Vert for ‖.
Of course, those notations should be only used among a group of people who know what they're doing otherwise it certainly make other readers confused.

We used to use "|" for inductive definitions, etc. We changed it because of
abs.
I think it is a good idea to use | . I will put that on my to-do list.
Please don't make that change now. I am about to commit a lot of changes to
the int library.
On Wed, Feb 25, 2015 at 2:18 PM, Soonho Kong<EMAIL_ADDRESS>wrote:

I think we should escape the character | as you suggested. I can do that
in the Lean server.
Thank you. I'll also work on emacs-side for that change.
Regarding the notation for absolute value, for your information, we have
some unicode characters such as

‖ (U+2016, double vertical line)
| (U+007C, vertical line)

and people can still define their own notations for abs and norm using
those unicode characters. Note that we already have the entries for them in
lean mode's input-method -- | for | and \Vert for ‖.
Of course, those notations should be only used among a group of people who
know what they're doing otherwise it certainly make other readers confused.
—
Reply to this email directly or view it on GitHub
https://github.com/leanprover/lean/issues/446#issuecomment-76034522.

@avigad I have integrated your latest commits.
I have replaced '|a|' with 'abs a', and used '(a | b)' for divides (btw, this is the same notation used in Coq).
Here the commit in my private repo:
https://github.com/leodemoura/lean/commit/3c24461e514f1bf262846066604f8ab9bdc064ba
I will add the new notation for recursive functions and inductive datatypes.
I will keep the old and new notations for a couple of months.

Thanks! Did you use some regexp search and replace to make the process less
painful?
Are you against the use of a different symbol, |, for either divides or
absolute value or both? Isabelle uses that solution for abs (as well as
"abs x"). I can live with (a | b), but all the parentheses look funny.

Did you use some regexp search and replace to make the process less
painful?

Yes, but I had to manually fix some cases.

Are you against the use of a different symbol, |, for either divides or
absolute value or both?

No, I'm not.  I'm fine with it.
I can change to unicode later.

I can live with (a | b), but all the parentheses look funny.

I think you referring to the (H : (a | b)) case, right?
I agree it looks weird.
I think the instances of the form (a | b) -> (c | d) -> ... look fine.

I will create a branch to experiment later.
It looks like the | binding is now to
U+2223 ∣ DIVIDES
which looks bad in my emacs. It is really skimpy.
The symbol
U+007C | VERTICAL LINE
looks much better.
http://en.wikipedia.org/wiki/Vertical_bar
http://www.fileformat.info/info/unicode/char/2223/index.htm
The fact that we were otherwise not having problems suggests that it can be
used for both abs and div, with
precedence |:55
reserve notation  | a:55 |
By the way, when I write
infix *** := foo
or
notation a *** b := foo a b
what are the default rbp's?
On Wed, Feb 25, 2015 at 6:44 PM, Leonardo de Moura<EMAIL_ADDRESS>
wrote:

Did you use some regexp search and replace to make the process less
painful?
Yes, but I had to manually fix some cases.
Are you against the use of a different symbol, |, for either divides or
absolute value or both?
No, I'm not. I'm fine with it.
I can change to unicode later.
I can live with (a | b), but all the parentheses look funny.
I think you referring to the (H : (a | b)) case, right?
I agree it looks weird.
I think the instances of the form (a | b) -> (c | d) -> ... look fine.
—
Reply to this email directly or view it on GitHub
https://github.com/leanprover/lean/issues/446#issuecomment-76086645.

The symbol
U+007C | VERTICAL LINE
looks much better.

Great!
It seems it is the standard symbol for abs. This is an extra plus.

what are the default rbp's?

It is 1. It can be easily changed in the code.
We have
#define LEAN_DEFAULT_PRECEDENCE 1

The symbol
U+007C | VERTICAL LINE
looks much better.

You're right. I'll make the change in lean-mode.

The symbol
U+007C | VERTICAL LINE
looks much better.
Great!

It seems it is the standard symbol for abs. This is an extra plus.

I think it's actually the ascii character '|' which we wanted to avoid.

It looks like the | binding is now to
U+2223 ∣ DIVIDES
which looks bad in my emacs. It is really skimpy.

@avigad, could you try the following trick? It helped @leodemoura and me before.

Install unicode-fonts package via M-x package-install command.

Add the following lines in your emacs setup:

(require 'unicode-fonts)
(unicode-fonts-setup)
```

I think it's actually the ascii character '|' which we wanted to avoid.

You are right :-)

When I try to install unicode-fonts I get "Error during download request:
Not Found". I will try again later.

Executing package-refresh-contents before installing unicode-fonts should help. If not, please let me know.

Yes, that solved it, and now the divides symbol | looks fine. Thanks!

@avigad, great! I've added those instructions in doc/emacs/README.md file via 88e287d.

For this particular case (type information in auto-complete), I can avoid the problem by only splitting a line at the first occurrence of | and accept the rest of the line as a type.

It's done by 05cfb21. I'm using an old version to test it:

Great!!!!

@soonhokong Thanks for finding the problem. I think we should escape the character | as you suggested.

@leodemoura, could you ping me when it's done. No rush please, of course :-)

@leodemoura, could you ping me when it's done. No rush please, of course :-)

Do we still need it? I believe only FINDP produces output mixing types and |. It is a sequence of the form
[name]|[type]

It is not ambiguous, since name never contains |. So, the solution you used is perfect.
In all other messages, we use | to separate an ID from the column and line numbers.
Am I missing the problem?

@leodemoura, you're right!


added question-answering part for Bengali language

Hi,
I just added the question answering mapping part for the Bengali language.
Please check and if it is eligible for merging please merge.
thanks and regards
Sagor

Hi @sagorbrur!
Thank you for your contribution! Would it be possible to provide some examples with shorter, simpler contexts? You might find some inspiration in some of the English examples.
Thank you once more!

Hi @osanseviero ,
Thank you.
There are no open-source QA datasets for Bengali except google's Tydiqa datasets.
The example I inserted here is actually from tydiqa dev datasets and these examples are benchmark examples.
If these large contexts make any problem for the model hub, I can change it.
Please let me know.
Regards

Thanks for explaining 🔥
If these are benchmark examples, I think they are fine then. I'll let @LysandreJik take a final look and then we can merge :)

Thank you for the contribution!

Thanks to you all.


bootstrap-switch doesn't work with Zepto (development branch)

With the move from jQuery to Zepto by @tomkwok, bootstrap-switch doesn't work anymore. There is a display problem where both ON and OFF labels are shown simultaneously (i.e. the "Publish" switch).
Should we go back to jQuery?

In my development branch, I've actually removed bootstrap-switch some time ago and replace those switches/toggles with normal checkboxes. Those toggles are confusing to me but a tick as an indicator is much clearer. Ghost use (styled) checkboxes from day one.

[ ]
[x]

@kabukky Do you think it's a good idea?

@tomkwok I see your point. A simple checkbox would indeed be clearer. The standard checkbox is just not pleasing to the eye. Maybe we need to style it a bit.
I'll revert to jquery and bootstrap-switch for now - I think using jquery for the admin interface is fine.
BTW, what is
<script type="text/javascript">$(function(){$.support.transition = false;});</script>

in admin.html for? You added that below the Bootstrap js file.

I added that to tell Zepto to tell Bootstrap's bootstrap.js (which requires jQuery/Zepto) to disable animation/transition/effects applied on expanding/collapsing of the hamburger menu (in the top-right corner) on mobile, or the expanded hamburger menu becomes non-collapsible (weird).

All right, I'll remove it then. Thanks for clearing it up :)


feat: add top-level WF + cli to upsert notion DB

Description
Will be handy primarily for structured data
Risk

Deploy Plan
queue bump: deploy connectors + restart notions

PMRW


Retrieving logo from MinIO fails with signature error

Is there an existing issue for this?

[X] I have searched the existing issues

Version of Corteza
2023.3.6
Current Behavior
After uploading a new logo file in the Admin-->User Interface Settings page, the logo does not display within Corteza.  Only a broken image placeholder is shown.
The file is properly uploaded to MinIO (both the <file>.jpeg and a <file>_preview.jpeg appear within the system folder in the bucket).  The file can be downloaded from MinIO with other clients.
On refreshing the page, the browser reports a 500 error when trying to load ui.mainLogo with response seeker can't seek
MinIO trace logs show a problem with the signature generation when trying to retrieve the newly-stored logo file.
<?xml version="1.0" encoding="UTF-8"?>
<Error>SignatureDoesNotMatch<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
</Error>

Expected Behavior
After uploading a logo, it should be stored on the MinIO instance and visible within Corteza.
Steps To Reproduce
No response
Environment and versions
Ubuntu 22.04
MinIO:
  - Standalone, pre-deployed instance
  - Version: 2023-05-18T00:05:36Z
  - Has Server Location/region set to custom value

Certain Corteza Env Settings:
  - MINIO_ENDPOINT=<host>:<path>
  - MINIO_SECURE=true
  - MINIO_BUCKET=<custom>
  - MINIO_PATH_PREFIX={component}/
  - MINIO_ACCESS_KEY=<secret>
  - MINIO_SECRET_KEY=<secret>

Anything else?
I did check that the MinIO keys and permissions work to download the file by using a separate script to download the file using the Authorization header in this form:
Authorization: AWS <access_key>:<signature>

But the MinIO trace log shows that Corteza is using an authorization method of AWS4-HMAC-SHA256:
Authorization: AWS4-HMAC-SHA256 Credential=***...
```

Other services that call MinIO and use the AWS4-style authorization header work fine.

I did some further exploring tonight and confirmed that it does seem to be a problem with how the signature is being calculated. Recomputing the AWS4-HMAC-SHA256 signature for the same request yields a different signature string for a working request than it does for the Corteza-generated request, even where the X-Amz-Date header is consistent between the two requests.
I tried working with some blobs in the Case Management app and noticed the same behavior if I uploaded binary data. But .txt files seem to download fine. I did note that there's an extra header coming across in those requests--an "If-Match" that looks like it's part of the signed headers. If I upload a copy of the same .txt file with a different extension (e.g., .bin), the "If-Match" header goes away and the file again refuses to download from MinIO, citing a signature calculation mismatch.


Optimize MongoDB adapter for edx-adapt

[x] Optimize DB structure:
[x] Extract logs into separate collection
[x] Add indexed
[x] Update MongoDB version

Now optimization need to be tested!

tested successfully
Could be closed


Customer case study blog post for Oct 2021

Publish a blog post summarizing recent case studies.

The majority of the check errors you are hitting are related to spaces at the end of paragraphs. Generally, linters (or systems that look for style or syntax in code bases) pick up a lot of spaces when it comes to markdown like this. I always double-check the end of my paragraphs before I do a commit, especially if I've been moving a lot of stuff around, to catch that before a linter does.
I've left some notes explaining the rest in the comments. If you want the full list of rules, I could make a list for you (or, if you want to read the comments in the actual config that do a decent job of explaining them, see https://github.com/pulumi/pulumi-hugo/blob/master/scripts/lint/lint-markdown.js).

ah, thank you! that's what I get for pasting this directly from google docs then :)


invalid environment variable values silently ignored

Output of restic version
restic 0.16.4 (v0.16.4-0-g3786536dc) compiled with go1.21.8 on linux/amd64
What backend/service did you use to store the repository?
local/any
Problem description / Steps to reproduce
restic seems to silently ignore problems with some environment variable parameters.
In my instance, the documentation wasn't clear that RESTIC_PACK_SIZE needed to be numeric only (and was in MB) and it silently ignored RESTIC_PACK_SIZE=64M and continued to use the default rather than issuing a warning or failing.
It does properly warn if the problem parameter is on the command line.
Expected behavior
I would expect restic to warn or fail if an environment variable it is using is not valid (as long as it is not being overridden on the command line).
Actual behavior
restic silently ignores problems with environment variables like RESTIC_PACK_SIZE.
Did restic help you today? Did it make you happy in any way?
Using it in a new situation, working pretty well!

This is unfortunately rather complex to properly fix. Currently, environment variables are parsed inside init() functions before the regular execution starts. At that point, the code has no idea which command will be called in the end. Blindly failing if the environment variable value leads to a few weird corner cases:

shell autocompletion will stop working if an invalid environment variable is set
environment variables for a different command can cause the current one to fail
RESTIC_PACK_SIZE=64M restic --pack-size 64 ... would still fail

Avoiding  these corner cases, however, requires restructuring how all commands are defined. And I'm not yet sure how a correct implementation would look like.

Hey @MichaelEischer
Do you think we can use something similar to what's present in the internal/backend/backend.go where there is an interface called ApplyEnvironmenter this interface has a method called ApplyEnvironment.
We enforce that all backend configurations must implement this method to read any required environment variables.
Do you think something similar can be done for the Command options, where we can enforce an order of reading environment variables, command options and a validate function?

Do you think something similar can be done for the Command options, where we can enforce an order of reading environment variables, command options and a validate function?

Adding something similar for the global options should be relatively easy. Extending that to more commands in a clean way isn't that easy however. While we could introduce some callback mess, I'd like to invert how the command setup works to get rid of all the involved global variables.
We currently have global variable with the options for each command along with another global variable that holds the cobra.Command. These are initialized in an init() function and therefore have no clean way of returning an error.
The clean way to implement this for the commands is to completely refactor the used pattern. The first part is that move the setup of the command flags into an Add(...) method on the options struct used by the command, see cmd/restic/cmd_key_add.go for an example. Next, the actual command creation also has to move into a NewKeyAddCmd function. The call then has to add the command to the parent command an so on. The call to New*Cmd could then also handle and verify the environment variables.
As a stopgap solution we could add a GlobalOptions.ApplyEnvironment() error method and call that from cmd/restic/main.go . That would at least take care of the problem here.


[BUG] Issues with Economies implementations and PicoJobs hooking

Describe the bug
The newer versions of the plugin are not detecting EconomyPlus.
To Reproduce
Steps to reproduce the behavior:

If I start the server, the plugin is detecting EconomyPlus, so the salary pay-out is not working.

Expected behavior
That it detects EconomyPlus.
Screenshots and Logs
If it not detects EconomyPlus:
2 economy implementations successfully registered!
If it detects EconomyPlus:
1 economy implementations successfully registered! ( It just detects only Vault)
Software Details:

Server Software: [e.g. Spigot]
Server Software Version [e.g. 1.17.1]
Plugin Version [e.g. v58e0ddd-DEV (Currently, because it works with this version)]
Java Version: [e.g.  Java JRE 16]

Additional context
Add any other context about the problem here.

A fix was implemented for this issue, please try with the latest DEV build. If this issue persists make sure to re-open this issue or create another one.


Delete DaemonSets for deleted NodeGroups

We should remove daemon sets  for the node group  that was removed from the NumaResourceOperator spec.

it seems we have still unanswered comments? perhaps a new version wasn't pushed?

it seems we have still unanswered comments? perhaps a new version wasn't pushed?

Sorry forgot to change the "ErrorS" comments.
I think all comments, but the one regarding the convenience to use a list of errors or not, have been addressed now.

thanks, looks ready now. Please restore the commit message text, which apparently got lost in the rebase/squash, and we can merge.


Move Combat to Server Side

Determining if someone is hit or killed should really be determined by the server, not the client. . .

e782b4625f4f1cdb76aa3a81075777e20a68826b


Fix postinstall script of detox on sh

Related to https://github.com/wix/detox/issues/627#issuecomment-388379155.
We can reproduce the issue by the following script:
$ sh -c "touch script.sh && chmod +x script.sh && script.sh"
# sh: script.sh: command not found

postinstall now uses node script to support Windows as well as Linux and macOS.
Please check if this solves your issue (as soon as a new release is published), If you still have problems, please report an issue/submit a PR.
In the meantime, I am closing this PR.


Green lighten to 1 is green

(new Colour('red')).lighten(1).rgb()
Object {r: 255, g: 255, b: 255}
(new Colour('blue')).lighten(1).rgb()
Object {r: 255, g: 255, b: 255}
(new Colour('green')).lighten(1).rgb()
Object {r: 0, g: 255, b: 0}

This is not actually a bug.
It looks strange because of HTML color spec and because lighten is a relative process.
First, HTML "red" and "blue" are both pure colors to start with (255 for one value and 0 for the others). HTML "green" is strangely only half brightness (128 for green, 0 for the others). If you were to use "lime" instead, you would get the values you want!
For more info on the lighten function being a relative process, see #53. Passing one basically doubles the brightness of the current value. Since "green" is half-bright, it takes that value up to 255 ("lime"). If you were to run (new Colour('green')).lighten(1).lighten(1).rgb() - you would also get the results you were expecting.

Interesting. Thanks for this explanation !


Change delegation settings modal

Instead of having a separate step to "reclaim voting power", we can keep that option in the modal constantly.
as per wireframes here:
https://xd.adobe.com/view/61a546b6-c405-41c8-a8a3-793ca84992e7-34f6/screen/52ce9474-6a13-42ae-9d65-fbc8d3e9b47f?fullscreen

Private Zenhub Image


Next steps

I'm trying this with a more complex project, and getting my head back around how the buildpack works. Some notes for next time:

All of my work so far is on the simplify_deploy branch.

This has been merged to master, so the readme reflects this fork, and not the official Heroku buildpack.

Most of the work is done in bin/steps/autoconfigure.

The autoconfigure script is called from bin/compile.

This is a really compelling project.

Notes here have been documented elsewhere.


posts/0000-01-02-YOUR-USERNAME.md

layout: slide
title: “Welcome to our second slide!”
Your test
Use the left arrow to go back!

Погано


[Suggestion] It would be better if add double quotes for the storage account name in the notification when disabling static website

OS: Win10
Build Version: 20210608.3
Repro Steps:

Right click a storage account -> Select "Disable Static Website" -> Click "Disable" when asking to disable static web hosting for the storage account.

Check the notification message.

Actual:
The storage account without double quotes.

Suggestion:
It would be better if add double quotes for the storage account name.

This issue has become stale and is at risk of being closed. The community has 60 days to upvote the issue. If it receives 5 upvotes we will keep it open and take another look. If not, we will close it. To learn more about how we handle issues, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this issue has received less than 5 community upvotes and we closed it. Still a big Thank You to you for taking the time to create it! To learn more about how we handle issues, please see our documentation.
Happy Coding!


Installation

Trying to download the checksums file and its signature from the installation guide does not work
$ curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt
The latest tag is pointing to a different place
https://github.com/operator-framework/operator-sdk/releases/tag/scorecard-kuttl%2Fv2.0.0

This breaks the instructions from this page https://sdk.operatorframework.io/docs/installation/#install-from-github-release

@gngeorgiev I was following the instructions from this page.
I believe that the problem is that the latest tag points to the wrong release (see link above).

That's correct. This is a script that works for specific versions in a Dockerfile:
ARG OPERATOR_SDK_VERSION="v1.5.0"
RUN export ARCH=$(case $(arch) in x86_64) echo -n amd64 ;; aarch64) echo -n arm64 ;; *) echo -n $(arch) ;; esac) && \
    export OS=$(uname | awk '{print tolower($0)}') && \
    export OPERATOR_SDK_DL_URL=https://github.com/operator-framework/operator-sdk/releases/download/${OPERATOR_SDK_VERSION} && \
    curl -LO ${OPERATOR_SDK_DL_URL}/operator-sdk_${OS}_${ARCH} && \
    gpg --keyserver keyserver.ubuntu.com --recv-keys 052996E2A20B5C7E && \
    curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt && \
    curl -LO ${OPERATOR_SDK_DL_URL}/checksums.txt.asc && \
    gpg -u "Operator SDK (release<EMAIL_ADDRESS>--verify checksums.txt.asc && \
    grep operator-sdk_${OS}_${ARCH} checksums.txt | sha256sum -c - && \
    chmod +x operator-sdk_${OS}_${ARCH} && mv operator-sdk_${OS}_${ARCH} /usr/local/bin/operator-sdk && \
    operator-sdk version

It's a better idea to use a version, rather than latest as the script suggests either ways.

Thanks @gngeorgiev!
So that probably means that the installation instructions need to change so they follow the same idea as your Dockerfile code above?

Yes, the instructions in https://sdk.operatorframework.io/docs/installation/ "Install from GitHub release" are just plain wrong.
Either make the reality match the docs or change the docs.

The plan is to set up a download resource on the SDK’s website to redirect to the correct release download links. Hoping to have this set up by Friday, and update the latest docs.

In the meantime, I've changed the non-binary-containing release as a pre-release so the doc link still works.


Set apache KeepAliveTimeout to greater than ELB idle timeout

One of the possible explanations for the 504 responses we're seeing
is that ELBs keep sending requests to connections closed by the app
webserver.
Recommendation we've received from AWS support is to set keep-alive
timeout to a value greater than ELB idle connection timeout (which
is 60s by default).
Looking at apache config on Elastic Beanstalk instances it doesn't
look like an explicit value is set anywhere, so Apache is probably
using the default KeepAliveTimeout of 5s.
Setting a value above 60s to see if it fixes the issue. There's a
risk that a high keep-alive value will tie up Apache workers with
idle connections if ELBs are opening more backend connections than
the number of Apache threads.
(http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/ts-elb-error-message.html#ts-elb-errorcodes-http504)

Do we have a plan for how we will test that this fixes the problem? Have we been able to recreate 504s ourselves?

@robyoung haven't been able to replicate 504s. I've tried disabling all request sources to staging and sending requests with different intervals, but it doesn't resolve in 504s. I couldn't find any explanation of how ELB decides to open a new backend connection vs reusing an existing one, so it's hard to come up with a pattern of requests that would trigger the issue.
On the other hand, we get a few of them in production every day for API, so a few days without any ELB 5xx should indicate that the problem is fixed.
The only possible problem with high keep-alives is if they lock Apache workers with idle connections, but that's something we can test by throwing a lot of simple requests to preview.

:+1: for a tryout on preview at least.


Add drop_last attribute to infinite batch samplers.

This PR was motivated by the fact that PL's deepspeed plugin expects the batch_sampler to have drop_last defined for some reason.

Do you know whether this change is sufficient to make it work?


Revert pydantic

Reverted pydantic from v2.4.0 to v1.10.12 for performance reason

Close as stale


Allow any expression for enum label values

This was messing up my spicy enum when formatting. It looks like:
type E = enum {
    A = '1',
    B = '2',
    C = '3',
    D = '4',
    E = '5',
    F = '6',
};

Just allowing any expression on the RHS of the equals seems right to me
Apologies if the generated files are a little off (like the alloc.h) - feel free to take a look at how that reacts locally

Merged with 74dd9c2b09c5e965c2f9fdf54d414a19eec734f6.


Support for executing bulk writes

As of MongoDB 3.2, db.collection now supports batch/bulk write operations (insert/update/remove): https://docs.mongodb.org/manual/reference/method/db.collection.bulkWrite/#db.collection.bulkWrite
Can support for this be added to the laravel-mongodb driver? This would greatly save on roundtrip performance for when there are a large number of batch updates in sequence.
I believe the PHP MongoDB driver already has support for executing bulk writes: http://php.net/manual/en/class.mongodb-driver-bulkwrite.php
Thanks!

Pull requests are appreciated :)

@davidchchang This is already possible, I'm already using the bulkwrite from version 3.2 in some imports scripts I wrote.
First I create a new connection
new Jenssegers\Mongodb\Connection(\Config::get('database.connections.mongodb'))
Then i select the collection like so: $this->collection = $this->connection->getMongoDB()->selectCollection('collection');
This will give me a MongoDB collection in return and on that collection I can call bulkWrite
Now this isn't using the this library, except for the connection, but should enable you to write bulk imports :)
And personally when importing in bulk I'd rather be as close to the driver as possible as to not cause any unnecessary overhead.

It will work great with bulk addition, this way, you just need to create on array and pass it to it.
$temp = [
            [
                'item'=> "envelopes"
            ],`enter code here`
            [
                'item'=> "envelopesas"
            ],
            [
                'item'=> "lala"
            ]
        ];

        $userData = DB::table('log')->raw( function ( $collection ) use ($temp) {

            return $collection->insertMany($temp);
        });

Assuming MyCol model defines mycol collection, shorter way to get MongoDB collection is this:
$collection = MyCol::query();

I tried maulikmmodi's suggestion and could not get it to work. It ran, didn't throw any errors... but nothing was added to DB

using php mongo library:https://github.com/mongodb/mongo-php-library
we need run :
composer require mongodb/mongodb
and then :
   ```
    use MongoDB\Client;

    $mongoClient=new Client();
    $mongodata=$mongoClient->mydb->mycol;
    $mongodata->insertMany($array);

How can i use updateMany?  any ideas guys

$temp = [             [                 'item'=> "envelopes"             ],             [                 'item'=> "envelopesas"             ],             [                 'item'=> "lala"             ]         ];
$userData = DB::connection('mongodb')->collection('your_collection')->raw( function ( $collection ) use ($temp ) {                 return $collection->insertMany($temp );             });
Works for me

using php mongo library:https://github.com/mongodb/mongo-php-library
we need run :
composer require mongodb/mongodb
and then :

use MongoDB\Client;

$mongoClient=new Client();
$mongodata=$mongoClient->mydb->mycol;
$mongodata->insertMany($array);

This is not using the bulkWrite as @davidchchang was asking, you are referring to the insertMany command.
In the end I just retrieved the underlying connection and used that.


Unload Cause BSOD

Question:
when i stop gbhv. it call DriverUnload, some times Event exits hanppen. then bsod.
information:
BSOD before:
windbg print:
Unknown exit reason! An exit was made but no handler was configured to handle it. Reason: 0x20()+0x39:(VMX_EXIT_REASON_EXECUTE_WRMSR )
fffff800`a98925f9 488b442438      mov     rax,qword ptr [rsp+38h]
!analyze -v
SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)
This is a very common bugcheck.  Usually the exception address pinpoints
the driver/function that caused the problem.  Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: ffffffffc0000096, The exception code that was not handled
Arg2: fffff800a98913ec, The address that the exception occurred at
Arg3: ffffd000e8f9f508, Exception Record Address
Arg4: ffffd000e8f9ed20, Context Record Address
FOLLOWUP_IP:
gbhv!ArchDisableVmxe+2c [e:\gbhv-master\gbhv\arch.c @ 100]
fffff800`a98913ec 0f22e0          mov     cr4,rax
Windbg code location:
FAULTING_SOURCE_CODE:
96:     // Enable the bit
97:     Register.VmxEnable = 0;
98:
99:     // Write it back to cr4

100:     __writecr4(Register.Flags);
101: }
102:

Hope
How to locate this problem. Can you provide some ideas?

Unloading is not currently working in the released version. What you are
seeing is due to the fact that the driverunload function cannot disable vmx
because it itself is running in the guest OS. To disable vmx, the driver
must forcibly exit to the root mode then disable vmx.
To resolve this, you can write a patch which, in driverunload, loads a
magic constant into a register then configures an instruction (like cpuid)
to cause a vmexit. During the exit handler, check the magic constant value.
If that constant is matched, then cause an IPI (or GenericDPCCall) and exit
vmx using the code currently in driverunload.
I don't have the time to currently add this feature, but it is a known
issue. In the mean time, restarting is the only true way to reload the
driver. (Not much of an issue if you're using VM snapshots)
On Sun, Apr 21, 2019, 11:21 PM NoHeart2019<EMAIL_ADDRESS>wrote:

Question:
when i stop gbhv. it call DriverUnload, some times Event exits hanppen.
then bsod.
information:
BSOD before:
windbg print:
Unknown exit reason! An exit was made but no handler was configured to
handle it. Reason: 0x20()+0x39:(VMX_EXIT_REASON_EXECUTE_WRMSR )
fffff800`a98925f9 488b442438 mov rax,qword ptr [rsp+38h]
!analyze -v
SYSTEM_THREAD_EXCEPTION_NOT_HANDLED (7e)
This is a very common bugcheck. Usually the exception address pinpoints
the driver/function that caused the problem. Always note this address
as well as the link date of the driver/image that contains this address.
Arguments:
Arg1: ffffffffc0000096, The exception code that was not handled
Arg2: fffff800a98913ec, The address that the exception occurred at
Arg3: ffffd000e8f9f508, Exception Record Address
Arg4: ffffd000e8f9ed20, Context Record Address
FOLLOWUP_IP:
gbhv!ArchDisableVmxe+2c [e:\gbhv-master\gbhv\arch.c @ 100]
fffff800`a98913ec 0f22e0 mov cr4,rax
Windbg code location:
FAULTING_SOURCE_CODE:
96: // Enable the bit
97: Register.VmxEnable = 0;
98:
99: // Write it back to cr4
100: __writecr4(Register.Flags);
101: }
102:
Hope
How to locate this problem. Can you provide some ideas?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Gbps/gbhv/issues/6, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABKZAKB3OYSQO2SU6G2NIRDPRUVMHANCNFSM4HHNEFCQ
.

Thanks  a lot

See #2

It's the noob proof feature :D


Lists scroll to the top every 3-5 seconds

When I scroll down the list over finished plots sorted by time finished to analyze the totalt time etc, the list automatically jumps to the top every 3-5 seconds. This is a bit annoying when you have scrolled down and found the plots of interest, and it just jumps to the top again.
Thanks for a great tool!

Are you sure it is not every 10 Seconds?
This issue is known (#51) and I don't know how to fix that properly with the GUI Library I use.
There is a Button to pause refresh on the top right. That should allow you to analyse the data in peace.


Add integration tests to ensure that error context data is being sent to clients (both LSP and legacy protocol)

(Parent issue #44897)
As of a42244f73b27256eca44def4daa552e91c2dbc55, the analyzer now generates context messages in some circumstances explaining why type promotion failed.  We should ensure that we have adequate integration tests to verify that these context messages are being delivered to analysis server clients.  We should test both the LSP protocol (used by VSCode) and the legacy protocol (used by IntelliJ).
Related: #44901, #44902.

I'm not sure who to assign this to--perhaps @bwilkerson?

@devoncarew do you think anyone will have a chance to work on this before the branch cut?

Do we not already have coverage here? This isn't about testing for the presence of the messages in general - that's covered by https://github.com/dart-lang/sdk/issues/45068 and the test.py work (https://github.com/dart-lang/sdk/issues/44905).
This is just to make sure that we're populating the information in the protocol messages to IDEs? @bwilkerson / @DanTup - is this already covered by existing tests?

There's a test in LSP here:
https://github.com/dart-lang/sdk/blob/3ce196935ee16081cf4710436ad50aaae2b4adb8/pkg/analysis_server/test/lsp/diagnostic_test.dart#L129-L144
It's not currently testing the specific text/location (it may be worth extending to do that), though it is ensuring there is a relatedInformation (which is where the context message shows up if I understand correctly).

I missed the "integration" part. I've opened https://dart-review.googlesource.com/c/sdk/+/193800/ that adds an LSP integration test that checks the contexts come through (unlike the test I noted above, these spawn the server in another process and then communicate with it exactly as a client would).


Load checkpoint to GPU but still exist in CPU

Prerequisite

[X] I have searched the existing and past issues but cannot get the expected help.
[X] I have read the FAQ documentation but cannot get the expected help.
[X] The bug has not been fixed in the latest version.

🐞 Describe the bug
I loaded the checkpoint to GPU.
But with my config still, 4GB model exist in CPU.
Environment
pytorch 1.9
Additional information
No response

This is due to the NVIDIA driver overhead caused by the first call of cuda in PyTorch.
Here's an example of a really tiny tensor before being moved to GPU:

Then after being moved to GPU:

The NVIDIA drivers use ~4GB of RAM. Here's a discussion here on PyTorch forums about it: https://discuss.pytorch.org/t/cpu-ram-usage-with-cuda-is-large-2-gb/117668

How can we avoid it or refresh it?

If you want to use a CUDA gpu, there is no avoiding this. This is just the price/overhead of using CUDA.

Thanks for your issue and experience sharing !
We have no experience with this at the moment, maybe you can ask MMDeploy, they have more experience with mobile deployment.


Union / Sum Datatype

Values can often be one of several datatypes (like string or int, class A or class B, ...).
There should be a way to model unions (anonymous or tagged).
Describing this in the schema is a bit tricky though, since the datatype property is just a single value.
Options:

allow the datatype property to be an array of possible types
add an  additional property for the Property  class:  somehting like alternativeDataTypes
require defining a custom datatype for each union.
There could be a UnionType class which has a datatypes property, and custom union datatypes specify "isA: "Union". That's rather awkward though, since you could end up with StringOrInt` etc datatypes , which pollute the readability of a schema quite a bit, and more importantly might be duplicated a lot

Non of these is really ideal.
Curios about alternative thoughts here.

Ah, an additional idea - which is actually kinda-sorta what I implemented in factordb:
Allow datatype to be a nested resource , although constrained to class types that refine existing datatypes with constraints.

The Union datatype can be useful, but it introduces a cost for data consumers: they will now need to perform type checking.
In programming languages like Rust, a Union type therefore does not exist by design. However, they do have enum values, which can definitely be used as Unions.
In atomic data, we do have enums, but these function more like JS enums (a list of options that do not contain values) instead of rust enums (a list of options that may contain any type of other value).
So I agree that currently, Atomic Data lacks a way to properly express something like Rust Enums or TS Unions.
Can you perhaps come up with a concrete real world example of something that you can't model right now with Atomic Data?