Upgrade 1.14.10 -> 1.15.4 network policies start dropping traffic

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
I upgraded from 1.14.10 to 1.15.4 following the upgrade instructions including the pre-flight checks that passed.
I am using a fairly minimal values.yaml for the cilium installation:
ipam:
  operator:
    clusterPoolIPv4PodCIDRList:
    - <IP_ADDRESS>/16

hubble:
  relay:
    enabled: true
  ui:
    enabled: true
serviceAccounts:
  cilium:
    name: cilium
  operator:
    name: cilium-operator
tunnelPort: 8473
tunnelProtocol: vxlan

I have an existing setup where I use network policies extensively. I know see a number of policies failing. One example is my jenkins server. Where I see a lot of traffic that is no longer being allowed. See the dropped.log file for details. This involves standard network policies for pod to pod communication not involving the host network, communication to the api server for which I have a cilium network policy.
The involved pod is:
> k get pods -n jenkins --show-labels
NAME                READY   STATUS    RESTARTS   AGE    LABELS
wamblee-jenkins-0   2/2     Running   0          143m   app.kubernetes.io/component=jenkins-controller,app.kubernetes.io/instance=wamblee,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=jenkins,apps.kubernetes.io/pod-index=0,controller-revision-hash=wamblee-jenkins-7f469f5cd6,statefulset.kubernetes.io/pod-name=wamblee-jenkins-0

There are no network policies in the kube-system namespace so access to the API server should be determined only based on policies in the jenkins namespace.
I have the feeling that matching of the pod labels is not working correctly. What does appear to be working is the default-allow-nothing rule (default deny policy). This causes all network policies that filter on the label not to match the jenkins pod so that the allow rule that applies is the allow nothing rule which causes all traffic to be denied.
For instance, here is my policy to allow traffic to the API server:
kind: CiliumNetworkPolicy
apiVersion: cilium.io/v2
metadata:
  name: jenkins-api-server-access
  namespace: jenkins
spec:
  endpointSelector:
    matchLabels:
      statefulset.kubernetes.io/pod-name: wamblee-jenkins-0
  egress:
    - toEntities:
        - all
      toPorts:
        - ports:
            - port: "6443"
              protocol: TCP

This policy is definitely working since when I delete it with cilium version 1.14.10, I see traffic from jenkins to the apiserver getting DROPPED, and the message disappear again when I apply the network policy again.
As said before there are many other networkpolicies that relate to the jenkins pod that are failing. They all use the same label selector for the jenkins pod. However, the above one is the easiest to troubleshoot since it can be reproduced using a single pod in a single namespace and a single networkpolicy.
Cilium Version
1.15.4
Kernel Version
Linux baboon 6.1.0-16-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.67-1 (2023-12-12) x86_64 GNU/Linux
Debian 12
Kubernetes Version
1.28.5
Regression
Yes, version 1.14.10 was working fine
Sysdump
cilium-sysdump-20240428-140900.zip
Relevant log output
> stdbuf -oL hubble observe -f | grep 'jenkins.*DROPPED'
Apr 28 11:47:02.544: jenkins/wamblee-jenkins-0:55218 (ID:21112) <> <IP_ADDRESS>:6443 (kube-apiserver) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:02.946: jenkins/wamblee-jenkins-0:52402 (ID:21112) <> <IP_ADDRESS>:6443 (kube-apiserver) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:03.952: jenkins/wamblee-jenkins-0:52402 (ID:21112) <> <IP_ADDRESS>:6443 (kube-apiserver) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:05.968: jenkins/wamblee-jenkins-0:52402 (ID:21112) <> <IP_ADDRESS>:6443 (kube-apiserver) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:08.683: exposure/httpd-wamblee-org-6d74cbcb5-mhbll:56978 (ID:36342) <> jenkins/wamblee-jenkins-0:8080 (ID:21112) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:09.707: exposure/httpd-wamblee-org-6d74cbcb5-mhbll:56978 (ID:36342) <> jenkins/wamblee-jenkins-0:8080 (ID:21112) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:10.224: jenkins/wamblee-jenkins-0:52402 (ID:21112) <> <IP_ADDRESS>:6443 (kube-apiserver) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:11.723: exposure/httpd-wamblee-org-6d74cbcb5-mhbll:56978 (ID:36342) <> jenkins/wamblee-jenkins-0:8080 (ID:21112) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:15.803: <IP_ADDRESS>:6443 (kube-apiserver) <> jenkins/wamblee-jenkins-0:57734 (ID:21112) Policy denied DROPPED (TCP Flags: ACK, FIN, PSH)
Apr 28 11:47:15.915: exposure/httpd-wamblee-org-6d74cbcb5-mhbll:56978 (ID:36342) <> jenkins/wamblee-jenkins-0:8080 (ID:21112) Policy denied DROPPED (TCP Flags: SYN)
Apr 28 11:47:18.416: jenkins/wamblee-jenkins-0:52402 (ID:21112) <> <IP_ADDRESS>:6443 (kube-apiserver) Policy denied DROPPED (TCP Flags: SYN)

Anything else?
No response
Cilium Users Document

[X] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

The upgrade command I used was
helm upgrade --install cilium cilium/cilium --version 1.15.4 \
  --set upgradeCompatibility=1.14 \
  --namespace kube-system --values values.yaml

When I detected the problem I was still using 'kube-apiserver' in the toEntities (which works with 1.14.10) and replaced it with 'all' for troubleshooting.

I did some additinoal testing on a single node k8s cluster running version k8s 1.29.2 and used cilium 1.15.4.
I created a dummy jenkins pod:
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: jenkins
    app.kubernetes.io/component: jenkins-controller
    app.kubernetes.io/instance: wamblee
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: jenkins
    apps.kubernetes.io/pod-index: "0"
    controller-revision-hash: wamblee-jenkins-7f469f5cd6
    statefulset.kubernetes.io/pod-name: wamblee-jenkins-0
  name: jenkins
  namespace: jenkins
spec:
  containers:
  - image: alpine
    name: jenkins
    command:
      - sh
      - -c
      - |
        apk add curl
        while :
        do
          curl -vk https://kubernetes.default.svc.cluster.local
          sleep 1
        done

The idea is that it continuously tries to connect to the api server and this will show 'Could not resolve host if the kubernetes API server lookup fails' and will also timeout in the later call to the api server it that is blocked as well.  Doing this, I quickly found out that only using the short label 'run: jenkins' actually worked so I used this to allow traffic for the API server using network policies.
I used the following three network policies
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: default-allow-nothing
  namespace: jenkins
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress
---
---
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: jenkins-allow-external-traffic
  namespace: jenkins
spec:
  podSelector:
    matchLabels:
      #run: jenkins
      #app.kubernetes.io/component: jenkins-controller
      #app.kubernetes.io/instance: wamblee
      #app.kubernetes.io/managed-by: Helm
      #app.kubernetes.io/name: jenkins

      # pod-index is not used by cilium
      #apps.kubernetes.io/pod-index: "0"
      #controller-revision-hash: wamblee-jenkins-7f469f5cd6
      statefulset.kubernetes.io/pod-name: wamblee-jenkins-0

  egress:
    - ports:
        - port: 53
          protocol: TCP
        - port: 53
          protocol: UDP
        - port: 80
        - port: 443
---
kind: CiliumNetworkPolicy
apiVersion: cilium.io/v2
metadata:
  name: jenkins-api-server-access
  namespace: jenkins
spec:
  endpointSelector:
    matchLabels:
      #statefulset.kubernetes.io/pod-name: wamblee-jenkins-0
      run: jenkins
  egress:
    - toEntities:
        - kube-apiserver
    - toPorts:
        - ports:
            - port: "6443"
              protocol: TCP

So, (1) a default allow nothing policy, (2) allowing DNS and some other protocols, (3) allowing access to the API server.
Then I tried using each of the labels in the jenkins pod one by one i n the network policy.
I found then that all labels worked apart from the last three: apps.kubernetes.io/pod-index, controller-revision-hash,  statefulset.kubernetes.io/pod-name. Now of course, not all of these labels would make sense to use in network policy in practice, but the last one does and I think it is not the job of the CNI to silently discard certain labels.
Is there a reason for this new behavior? I could not find any documentation for this on the k8s website and it has definitely changed in version 1.15.4 compared to 1.14.10.  I also double-checked this test setup with 1.14.10 and that version still works as expected.

I think I have found it based on the 'identity relevant labels' documented at https://docs.cilium.io/en/stable/operations/performance/scalability/identity-relevant-labels/.
There has been an extension of these labels in version 1.15.4. I think in any case three things should be done:

a validation should be done in the pre-flight checks. These should fail when these labels are used.
a validating webhook should reject network policies that use these labels
users should be informed on the release notes that the set of identity relevant labels has been extended

Is your issue similar to #30073?

Hi @ErikEngerd, it does look likely that this is related to the PR linked above (#28003). As @joestringer noted there, that really should have been called out as a bigger change, and mentioned in the upgrade guide, as you say.
I've marked this for the attention of SIG Policy, so they can prioritize accordingly.

This is precisely the issue indeed


Report inconsistency in the command MSET

This issue is all about ensuring we are as close to Redis as possible. The command in focus for this issue is MSET.
Go through the official documentation of the command MSET on Redis and identify the inconsistencies. The inconsistencies could be in

unhandled edge case
unexpected behavior
unsupported option

Because we are trying to be compatible with Redis v7.2.5, I would recommend you try out different variants of the command with different inputs on that specific version. The instructions on running Redis v7.2.5 locally

from source code
use Docker

Once you find the discrepancy, you can either

raise an issue on Dice repository with details, or
try to fix it yourself and raise a PR

If you are raising the issue, make sure you provide the details such as

use the template and provide the following details
steps to reproduce (series of commands)
observed output on DiceDB
observed output on Redis v7.2.5

Also, feel free to update the documentation and raise the PR in the docs repository.

You will need to go deeper into the command make sure you are covering all cases and reporting the inconsistencies or fixing them. The deeper the work, the better our stability will be. Also, it is possible that we do not find any discrepancies, so please mention the same in the comment on this issue. Mention the PR or issue links that you create under this issue.

Hi @arpitbbhayani , can you please assign this issue to me. I'll take a look. Thanks

@Aditya-Bhalerao go for it :)

Update:
I found an inconsistency with the MSET command and have reported it here: #406
I’m still working on finding more inconsistencies. Will update as I progress.

Hello @Aditya-Bhalerao,
There has been no activity on this issue for the past 5 days.
It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.
We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.
Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.
Thanks again.

 I will update with my findings soon.

Update:
Found an inconsistency and have reported it here: https://github.com/DiceDB/dice/issues/516

Hi @arpitbbhayani,
Implementation-wise, there is one difference:
Redis:
In Redis, the process of encoding values and setting them in the database is handled in a single loop. During each iteration:

The value is encoded
Immediately after, the key and the encoded value are set in the database

Example:
for (j = 1; j < c->argc; j += 2) {
    c->argv[j+1] = tryObjectEncoding(c->argv[j+1]);
    setKey(c, c->db, c->argv[j], c->argv[j + 1], setkey_flags);  // Setting the key in the database
}

DiceDB:
In DiceDB, the process is different:

The first loop encodes values and stores key-value pairs in a map, this map is then passed to an intermediate function
Inside the intermediate function, a second loop iterates over the map and calls the key-setting function for each entry

Example:
// First loop: encoding and populating the map
for i := 0; i < len(args); i += 2 {
    key, value := args[i], args[i+1]
    oType, oEnc := deduceTypeEncoding(value)
    insertMap[key] = store.NewObj(value, exDurationMs, oType, oEnc)
}

// Intermediate function: PutAll
func PutAll(data map[string]*Obj) {
    for k, obj := range data {
        Put(k, obj)  // Setting the key in the database
    }
}

It is unclear why DiceDB separates these operations into two loops, as this adds unnecessary overhead by creating an intermediate map and performing multiple loops.
Is there any particular reason for this two-step process? If not, can I go ahead and modify the implementation to handle both encoding and key-setting in a single loop without using the intermediate map?
I've already tried and tested a few commands with this approach, but I need to thoroughly check it further.

Apart from above filed, no other inconsistencies found in the command.

Hello @arpitbbhayani ,
Can you assign this issue to me ?

Hello @Aditya-Bhalerao,
There has been no activity on this issue for the past 5 days.
It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.
We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.
Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.
Thanks again.

Apart from above reported, no other inconsistencies found in the command.

@arpitbbhayani, This issue can be closed. I have filed all inconsistencies which I found.

Hello @Aditya-Bhalerao,
There has been no activity on this issue for the past 5 days.
It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.
We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.
Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.
Thanks again.

Hello @arbha1erao,
There has been no activity on this issue for the past 5 days.
It would be awesome if you keep posting updates to this issue so that we know you are actively working on it.
We are really eager to close this issue at the earliest, hence if we continue to see the inactivity, we will have to reassign the issue to someone else. We are doing this to ensure that the project maintains its momentum and others are not blocked on this work.
Just drop a comment with the current status of the work or share any issues you are facing. We can always chip in to help you out.
Thanks again.


[Code Addition Request]: Add a new project

Have you completed your first issue?

[X] I have completed my first issue

Guidelines

[X] I have read the guidelines
[ ] I have the link to my latest merged PR

Latest Merged PR Link
https://github.com/UTSAVS26/PyVerse/pull/900
Project Description
Semi-Supervised Image Classification
The main idea is to use a consistency regularization approach to leverage the unlabeled data for training. The process involves:

Training a neural network on a small labeled dataset.
Adding consistency regularization to utilize the large unlabeled dataset by enforcing that the model's predictions for augmented versions of the same image should be similar.

Use the CIFAR-10 dataset, which consists of 10 different classes of images. Split the dataset into labeled and unlabeled subsets. For example, use only 10% of the dataset as labeled and the remaining 90% as unlabeled. The model will be trained to make similar predictions for different augmented versions of the same image (e.g., using random cropping, flipping, etc.).
Full Name
Dharun Kumar
Participant Role
Contributor in GSSOC'EXTD and Hacktoberfest

@Dharun235 raise 1 issue at a time.

@UTSAVS26 Okay.
Thanks.


minutes-grid-increment results in undefined

Describe the bug
I get an error when using minutes-grid. I found this issue, but it looks like this hasn't been fixed. Or is it not been deployed yet? As far as I could see it was tagged in the 5.3.0 version.
Related issue: https://github.com/Vuepic/vue-datepicker/issues/401
To Reproduce
Steps to reproduce the behavior:

Open the link (see "Screenshots")
It should open Playground.vue
Click on the time picker, it will open
Click on the minutes-button, it results in this error (in console):

@vuepic_vue-datepicker.js?v=1f8080ac:7732 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'value')

Expected behavior
No error 😄
Screenshots
I've made a small Stackblitz: https://stackblitz.com/edit/vuepic-vue-datepicker-wrtjqj?file=src%2Fcomponents%2FPlayground.vue
Desktop & mobile (please complete the following information):
I am using v5.3.0:
  '@vuepic/vue-datepicker':
    specifier: ^5.3.0
    version<EMAIL_ADDRESS>

The fix is included in the released v6

Thank you very much @Jasenkoo! It works like a breeze! Upgrading was also very easy, thank you!


style: make sat buttons text a bit smaller

The text does not fit in one line on windows
Type of change (Remove other not matching type)

Bug fix (non-breaking change which fixes an issue)

Screenshots of the changes (If any) -
Problem:

needs to be tested on windows.
How Has This Been Tested?
open the LNURL pay screen by sending to<EMAIL_ADDRESS>

If we want smaller sat buttons, I think we should make a "small" variation of the button instead.
(Rather than creating side-effects by settings a smaller font-size on the parent)

+1 this was more a quick hack, not knowing what I do to give Amy something to make the videos not look that broken :)

Is this fixed now by #850 ?

I think so.


The container does not work with MySQL

Describe the bug
The docker container doesn't work when configured to use mysql.

Apparently label-studio doesn't come with the python client for mysql because the error is displayed is ModuleNotFoundError: No module named 'MySQLdb'

When the client is installed before the service starts with pip3 install mysqlclient && label-studio start, the migrations do not finish.

Even after executing the command ALTER DATABASE 'labelstudio' CHARACTER SET utf8; in the database, the same error django.db.utils.OperationalError: (1071, 'Specified key was too long; max key length is 3072 bytes') is shown.

To Reproduce
Steps to reproduce the behavior:

Configure the docker-compose file to use mysql
run docker-compose up

Expected behavior
Just to run with mysql
Screenshots 1
app_1  | ./deploy/docker-entrypoint.sh: Looking for init scripts in /label-studio/deploy/docker-entrypoint.d/
app_1  | ./deploy/docker-entrypoint.sh: Launching /label-studio/deploy/docker-entrypoint.d/10-copy-static-data.sh
app_1  | => Copy static data to a shared folder...
app_1  | => Successfully copied.
app_1  | ./deploy/docker-entrypoint.sh: Launching /label-studio/deploy/docker-entrypoint.d/20-wait-for-db.sh
app_1  | => Waiting for MySQL...
app_1  | => MySQL is up.
app_1  | ./deploy/docker-entrypoint.sh: Launching /label-studio/deploy/docker-entrypoint.d/30-run-db-migrations.sh
app_1  | => Do database migrations...
app_1  | => Database and media directory: /label-studio/data
app_1  | => Static URL is set to: /static/
app_1  | Starting new HTTPS connection (1): pypi.org:443
app_1  | https://pypi.org:443 "GET /pypi/label-studio/json HTTP/1.1" 200 46133
app_1  | Traceback (most recent call last):
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/mysql/base.py", line 15, in <module>
app_1  |     import MySQLdb as Database
app_1  | ModuleNotFoundError: No module named 'MySQLdb'
app_1  |
app_1  | The above exception was the direct cause of the following exception:
app_1  |
app_1  | Traceback (most recent call last):
app_1  |   File "/label-studio/label_studio/manage.py", line 22, in <module>
app_1  |     execute_from_command_line(sys.argv)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
app_1  |     utility.execute()
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 377, in execute
app_1  |     django.setup()
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/__init__.py", line 24, in setup
app_1  |     apps.populate(settings.INSTALLED_APPS)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/apps/registry.py", line 114, in populate
app_1  |     app_config.import_models()
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/apps/config.py", line 211, in import_models
app_1  |     self.models_module = import_module(models_module_name)
app_1  |   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
app_1  |     return _bootstrap._gcd_import(name[level:], package, level)
app_1  |   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
app_1  |   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
app_1  |   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
app_1  |   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
app_1  |   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
app_1  |   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/contrib/auth/models.py", line 2, in <module>
app_1  |     from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/contrib/auth/base_user.py", line 48, in <module>
app_1  |     class AbstractBaseUser(models.Model):
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/models/base.py", line 122, in __new__
app_1  |     new_class.add_to_class('_meta', Options(meta, app_label))
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/models/base.py", line 326, in add_to_class
app_1  |     value.contribute_to_class(cls, name)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/models/options.py", line 206, in contribute_to_class
app_1  |     self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/__init__.py", line 28, in __getattr__
app_1  |     return getattr(connections[DEFAULT_DB_ALIAS], item)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/utils.py", line 214, in __getitem__
app_1  |     backend = load_backend(db['ENGINE'])
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/utils.py", line 111, in load_backend
app_1  |     return import_module('%s.base' % backend_name)
app_1  |   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
app_1  |     return _bootstrap._gcd_import(name[level:], package, level)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/mysql/base.py", line 17, in <module>
app_1  |     raise ImproperlyConfigured(
app_1  | django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module.
app_1  | Did you install mysqlclient?
app_1  | Sentry is attempting to send 2 pending error messages
app_1  | Waiting up to 2 seconds
app_1  | Press Ctrl-C to quit
classipii_app_1 exited with code 1

Screenshots 2
app_1  | Starting new HTTPS connection (1): pypi.org:443
app_1  | https://pypi.org:443 "GET /pypi/label-studio/json HTTP/1.1" 200 46133
app_1  | Initializing database..
app_1  | Traceback (most recent call last):
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 84, in _execute
app_1  |     return self.cursor.execute(sql, params)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/mysql/base.py", line 73, in execute
app_1  |     return self.cursor.execute(query, args)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/MySQLdb/cursors.py", line 206, in execute
app_1  |     res = self._query(query)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/MySQLdb/cursors.py", line 319, in _query
app_1  |     db.query(q)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/MySQLdb/connections.py", line 254, in query
app_1  |     _mysql.connection.query(self, query)
app_1  | MySQLdb._exceptions.OperationalError: (1071, 'Specified key was too long; max key length is 3072 bytes')
app_1  |
app_1  | The above exception was the direct cause of the following exception:
app_1  |
app_1  | Traceback (most recent call last):
app_1  |   File "/usr/local/bin/label-studio", line 33, in <module>
app_1  |     sys.exit(load_entry_point('label-studio', 'console_scripts', 'label-studio')())
app_1  |   File "/label-studio/label_studio/server.py", line 283, in main
app_1  |     _apply_database_migrations()
app_1  |   File "/label-studio/label_studio/server.py", line 59, in _apply_database_migrations
app_1  |     call_command('migrate', '--no-color', verbosity=0)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/core/management/__init__.py", line 168, in call_command
app_1  |     return command.execute(*args, **defaults)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 371, in execute
app_1  |     output = self.handle(*args, **options)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/core/management/base.py", line 85, in wrapped
app_1  |     res = handle_func(*args, **kwargs)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/core/management/commands/migrate.py", line 243, in handle
app_1  |     post_migrate_state = executor.migrate(
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/migrations/executor.py", line 117, in migrate
app_1  |     state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
app_1  |     state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/migrations/executor.py", line 227, in apply_migration
app_1  |     state = migration.apply(state, schema_editor)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/migrations/migration.py", line 124, in apply
app_1  |     operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/migrations/operations/models.py", line 808, in database_forwards
app_1  |     schema_editor.add_constraint(model, self.constraint)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/base/schema.py", line 362, in add_constraint
app_1  |     self.execute(sql)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/base/schema.py", line 142, in execute
app_1  |     cursor.execute(sql, params)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/sentry_sdk/integrations/django/__init__.py", line 544, in execute
app_1  |     return real_execute(self, sql, params)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 66, in execute
app_1  |     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
app_1  |     return executor(sql, params, many, context)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 84, in _execute
app_1  |     return self.cursor.execute(sql, params)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/utils.py", line 90, in __exit__
app_1  |     raise dj_exc_value.with_traceback(traceback) from exc_value
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/utils.py", line 84, in _execute
app_1  |     return self.cursor.execute(sql, params)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/django/db/backends/mysql/base.py", line 73, in execute
app_1  |     return self.cursor.execute(query, args)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/MySQLdb/cursors.py", line 206, in execute
app_1  |     res = self._query(query)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/MySQLdb/cursors.py", line 319, in _query
app_1  |     db.query(q)
app_1  |   File "/usr/local/lib/python3.8/dist-packages/MySQLdb/connections.py", line 254, in query
app_1  |     _mysql.connection.query(self, query)
app_1  | django.db.utils.OperationalError: (1071, 'Specified key was too long; max key length is 3072 bytes')
app_1  | Sentry is attempting to send 2 pending error messages
app_1  | Waiting up to 2 seconds
app_1  | Press Ctrl-C to quit
classipii_app_1 exited with code 1

Environment (please complete the following information):

OS: [windows]
Label Studio Version [latest]

Additional context
Add any other context about the problem here.

Unfortunately LS doesn't support mysql officially.
Similar issue: https://github.com/heartexlabs/label-studio/issues/1896
You can try to install it using pip install label-studio[mysql].

now can't install label_studio[mysql] ,when i try it return:  'no matches found: label-studio[mysql]'

pip install label-studio[mysql]


Recipe pass master issue

Featuring #351, #347, and anything else anyone cares to say:

Akashic ligatures shouldn't be more expensive than the bookshelves

Foci shouldn't be that expensive


Fixed :: use case when adding casted values

This is to fix an issue where parentheses were needed when using :: in operations such as:
select '4' :: integer + 5;
and
select 5 :: string || ' apples'

can you add some test cases for negative values in CastFunctionTest?

Current coverage is 81.47% (diff: 100%)

Merging #4668 into master will decrease coverage by 0.16%

@@             master      #4668   diff @@
==========================================
  Files          1307       1307
  Lines         44956      44956
  Methods           0          0
  Messages          0          0
  Branches       7072       7072
==========================================
- Hits          36703      36627    -76
- Misses         5563       5636    +73
- Partials       2690       2693     +3

Powered by Codecov. Last update 7765874...4c62123


get-pipx.py fails if ~/.local/pipx does not exist

$ curl https://raw.githubusercontent.com/cs01/pipx/master/get-pipx.py -fsSL | python3 - --no-modify-path
Traceback (most recent call last):
  File "<stdin>", line 221, in <module>
  File "<stdin>", line 193, in main
  File "/home/neersighted/.pyenv/versions/3.7.0/lib/python3.7/pathlib.py", line 1230, in mkdir
    self._accessor.mkdir(self, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/neersighted/.local/pipx/venvs'

Oh no! Thanks for reporting, I’ll get it fixed.

Should  be fixed now. You can try it out with
pipx upgrade pipx


MimeType 'audio/wav' since no more supported by Safari Preview 117

When testing the polyfill in Safari 14.01:
MediaRecorder.isTypeSupported('audio/wav') --> true
MediaRecorder.isTypeSupported('audio/mp4') --> false
When testing the polyfill in Safari 14.1 (SAfari preview 117):
MediaRecorder.isTypeSupported('audio/wav') --> false
MediaRecorder.isTypeSupported('audio/mp4') --> true

It seems as though Safari 14.0.3 now has MediaRecorder API support with audio/mp4 support, but not audio/wav. I'm still using this library to polyfill this version of Safari because it seems like there's issues with the stop/dataavailable event fire order that prevents us from getting the last chunk of data.


GH-3385-jackson-upgrade

Signed-off-by: Bart Hanssens<EMAIL_ADDRESS>GitHub issue resolved: #3385
Briefly describe the changes proposed in this PR:

bump jackson version (fixing a jackson CVE issue)

PR Author Checklist (see the contributor guidelines for more details):

[x] my pull request is self-contained
[ ] I've added tests for the changes I made
[x] I've applied code formatting (you can use mvn process-resources to format from the command line)
[x] I've squashed my commits where necessary
[x] every commit message starts with the issue number (GH-xxxx) followed by a meaningful description of the change

I'll try to back port this one too RDF4J 4.0.x since I'm anyways going to back port another fix.


Add sepolicy to allow thermal daemon to set property

Add sepolicy rules to allow thermal daemon to set property.
Tracked-On: OAM-115389

Android CI has started Engineering Build for this issue ,Please check the linked Tracked-On issue/Android CI Web  for more details.

Android CI has completed Engineering Build for this issue, build is FAILURE. Please check the linked Tracked-On issue/Android CI Web  for more details

Android CI has started Engineering Build for this issue ,Please check the linked Tracked-On issue/Android CI Web  for more details.

Android CI has started Engineering Build for this issue ,Please check the linked Tracked-On issue/Android CI Web  for more details.

Android CI has started Engineering Build for this issue ,Please check the linked Tracked-On issue/Android CI Web  for more details.

Android CI has completed Engineering Build for this issue, build is FAILURE. Please check the linked Tracked-On issue/Android CI Web  for more details

Android CI has completed Engineering Build for this issue, build is SUCCESS.To merge the changes please click on APPROVE FOR MERGE button in Android CI WEB


Rari Fuse Mainnet QA

Review Info

Description
Info

Subgraph Reviewed
https://subgraphs.messari.io/subgraph?endpoint=https://api.thegraph.com/subgraphs/name/messari/rari-fuse-ethereum&tab=protocol

Schema Version
2.0.1

Subgraph Version
1.1.3

Methodology Version
1.0.0

Evidence Spreadsheet
https://docs.google.com/spreadsheets/d/1Qd_4aTM2ShSjIQ7SAnS4PYMSxlj1eEVe/edit?usp=sharing&ouid=104027705798814439445&rtpof=true&sd=true

Metric to Review
Protocol TVL
As compared to official website data, both subgraph and Defi Llama data show a large discrepancy of around 50%. This maybe due to the inactive pools that are still being tracked by the subgraph and Defi Llama.

Subgraph
Defi Llama
Absolute Difference
% Difference
Official Website
Absolute Difference
% Difference

196,250,765
192,916,184
3,334,581
1.73%
127,067,102
69,183,662
54.45%

Pool Overview TVL

6 out of top 20 pools in the subgraph data are not shown in the official website.
Out of 14 pools that are shown in the official website, only two pools have large discrepancies compared to subgraph data ( The 11th and 20th pools)
Only one out of the six missing pools have accurate TVL subgraph data as compared against to DeBank data.
Please refer to the sheet "Pool_Overview" for details

TotalBorrowBalance
We have computed the revenue implied by the data from official website on 30 August 2022 and this shows large discrepancy with subgraph data. Refer to the sheet "Support-BorrowBalance". This is maybe due to the reason that some of the pools that are being tracked by the subgraph are not displayed on the official website.

Subgraph
Official Website
Absolute Difference
% Difference

211,624,071
91,671,114
119,952,957
130.85%

Rates
The interest rates in subgraph is slightly off compared to the official website. E.g. for Ferrari Pool,

Subgraph
Official Website

Borrowing APR
2.86%
2.96%

In addition, this pool's name "FeiRari" per the official website, is "Tetranode's Flavor of the Month Fei Protocol" in the subgraph.
Editorials
There are some differences we have noticed and are not necessarily data errors.

EVENT: Some of the latest withdraw transactions in Tenderly are missing in the subgraph data. Some of them are also not classified as withdraw transactions in Tenderly. Four out of 10 repay transactions in the subsgraph data are not classified as repay transactions in Tenderly.

POSITION: The ranking in balance column is not sorted by value. The hashes in the 'repays' column cannot be copied and opened in Etherscan. The hash of the third position in the 'borrows' column cannot be copied and opened in Etherscan too. It also displays error, see the screenshots below

Please refer to the attached worksheet for details.

Notes:

[ ] Need to make mainnet use the price calc switch

@dmelotik The only issue is dailydepositUSD spike on jan 14 2022. everything else looks good. The spikes on April 30 2022 on borrow and revenue are due to the fei hack.

dailydepositUSD issue resolved
https://twitter.com/BlockSecTeam/status/1520350965274386433?s=20&t=UQ6FVaxg-3Sd7weIH3rUZA
rentrance hack led to huge spike

@dmelotik added the arbitrum issue here. these two pools should be blacked out.

this issue was resolved. arbitrum rari fuse good to go.


add annotation to csv to make storageclaim internal

add the annotation operators.operatorframework.io/internal-objects to the operator csv to make storageClaim internal/ to remove it from the console
Ref: https://docs.openshift.com/container-platform/4.15/operators/operator_sdk/osdk-generating-csvs.html#osdk-hiding-internal-objects_osdk-generating-csvs

/cherry-pick release-4.16


Run mimalloc-test result not all free

$ mimalloc-test.exe
heap stats:     peak      total      freed       unit      count
  reserved:   256.0 mb   256.0 mb       0 b        1 b              not all freed!
 committed:   261.0 kb   261.0 kb       0 b        1 b              not all freed!
     reset:       0 b        0 b        0 b        1 b              ok
   touched:       0 b        0 b        0 b        1 b              ok
  segments:       1          1          0                           not all freed!
-abandoned:       0          0          0                           ok
   -cached:       0          0          0                           ok
     pages:       4          4          0                           not all freed!
-abandoned:       0          0          0                           ok
 -extended:       0
 -noretire:       0
     mmaps:       0
   commits:       5
   threads:       0          0          0                           ok
  searches:     0.0 avg
numa nodes:       1
   process: user: 0.000 s, system: 0.031 s, faults: 1235, reclaims: 0, rss: 4.5 mb, commit charge: 1.9 mb

Thanks @sniper00; this is expected -- usually the C runtime or other things still have things allocated by the time the statistics are printed. One way to get around this is to uncomment the line // mi_stats_reset(); which resets the statistics at the start of main so earlier allocations are ignored.
https://github.com/microsoft/mimalloc/blob/71160e2bac443c0dd35c7ee13993466efcee57b2/test/test-stress.c#L245
Also, you may need to uncomment // mi_collect(true); right before printing the statistics to ensure everything is fully deallocated.
This is enough on Windows, but under pthreads that still will leave some memory allocated; if you build in debug mode you can see detailed statistics, which includes:
heap stats:     peak      total      freed       unit      count
normal   1:     9.6 mb    14.6 mb    14.6 mb       8 b      1.9 m   ok
normal   4:    38.2 mb    57.6 mb    57.6 mb      32 b      1.8 m   ok
normal   6:    57.6 mb    87.2 mb    87.2 mb      48 b      1.9 m   ok
normal   9:    95.5 mb   144.9 mb   144.9 mb      80 b      1.8 m   ok
normal  13:   191.0 mb   288.8 mb   288.8 mb     160 b      1.8 m   ok
normal  17:   154.0 mb   154.4 mb   154.4 mb     320 b    506.2 k   not all freed!
normal  21:   305.7 mb   305.7 mb   305.7 mb     640 b    500.9 k   ok
normal  23:    19.8 mb    28.2 mb    28.2 mb     896 b     33.0 k   ok.
...

Here we see that some  320b blocks are leaking -- these are pthread metadata which is allocated on pthread creation but not freed (I think these are cached).
Hope this helps.

@daanx Ok, thanks.


Provide options to enable/disable the removal of decorators and replacement of resources in a prod build with AOT

This is a feature request referring to the prod build with AOT using the CLI. A more general title would be "Provide options to make the applied transformations configurable". For my use case, it is sufficient to just consider the two specific options described below - but other developers might have other use-cases.
Requirements
It would be very helpful to me and many other developers if the compiler options would provide means to control the removal of decorators in a prod build with AOT. There are two ways that could make it happen. Either in tsconfig.json as a parameter to angularCompilerOptions, e.g.:
  "angularCompilerOptions": {
    "aotRemoveDecorators": false
  }

Or as a switch to the ng-command, e.g.
ng build --prod --aotRemoveDecorators=false
The wording of such an option is yet to be discussed. I personally would favor to distinguish options that are only valid for either AOT or JIT by having a prefix/suffix in the option's name as indicated.
Additional Requirement
For the use-case described below, it is also important to replace style and template URLs by their content. This is what is done in JIT mode just a few lines above the referenced code (see link below). Now either the proposed option implies this behavior (which I don't like since it is not obvious) or another option gives the developer control for that behavior, too, e.g.
ng build --prod --aotRemoveDecorators=false --aotReplaceResources=true
Use-Case
There are Angular apps that use standard Angular means. To make those applications run fast, it makes sense to use AOT. However, many of those applications also have a need to build Angular components dynamically and invoke the JIT compiler. I am aware that it seems unusual to use JIT and AOT in one app, but it is also a reality that many applications have this need (just do a search for AOT and JIT in the angular github issues). It has also been shown that it is indeed possible to make AOT and JIT work together (e.g. see https://github.com/angular/angular/issues/15510). However, since the CLI removes the decorators in a prod build, the JIT compiler cannot compile the components and is emitting errors like "Please add a @NgModule annotation".
To make it work nowadays, it requires to hand-craft the build yourself using webpack or another bundler, but it is much more complex. Instead, it would be much more convenient to rely on the Angular CLI and use the powerful ecosystem that the CLI provides. The cost of adding such an option seems to me negligible compared to the pain many developers have. And I would highly appreciate if the CLI team would consider it.
The code responsible for the removal
The code responsible for the removal is located here: https://github.com/angular/angular-cli/blob/5f73a75193bc1217cbfb7f2240355679517cc8d0/packages/%40ngtools/webpack/src/angular_compiler_plugin.ts#L682
Workarounds
I don't know any workaround and I would be very happy to hear some. For example, I was wondering if it is possible to override the function "removeDecorators" to just do nothing? Any other ideas?

Put import { NgModule } from './ngmodule.decorator'; in desired file where you want to preserve decorator

Thanks, @p3x-robot . It's funny, we do the same workaround as you but with Gradle (which is our main build invoking the CLI - not going into details why) just after npm install. It is renaming removeDecorators by replaceResources in angular_compiler_plugin.js. One could also think of applying such a hack as part of the postinstall hook in packages.json if not using gradle. However, this is very unstable meaning that it can break anytime the CLI changes.
This is why I hope that @ocombe , besides being busy with many other tasks, can eventually help to solve this issue. In my view, he understands the pain and the use-cases described in this feature request.
@p3x-robot : with our help, others should now be able to figure out what to do. I highly appreciate that you shared it here! If you ask me, we should let the Angular guys do their job and avoid adding further comments.

Could the commit of marcelamsler be merged on angular-cli? I need also to skip removal of decorators in aot mode :(

In my view, the commit is only 50% correct. For JIT to work, the resources must also be replaced which is the IF condition just above where @marcelamsler made his change where function 'replaceResources' is called. Therefore, instead of having a narrow argument like 'skipRemoveDecorators', maybe a more general like 'enableJitInAot' might provide a higher level of abstraction.

Yep
I also noticed the build-optimizer option of angular-cli AOT compilation (default: true since Angular 5), is stripping the decorators as well :( so workaround for me, is to apply the hack above, and also to set build-optimizer to false. Definitively too much.....

I am sorry my pull-request for this commit was a mistake. I just tried if it could work but it did not. I am planning to do it, but I don't have time at the moment.
My current work-around is to disable build-optimizer (ng-build --aot --build-optimizer=false) and to re-export the needed decorators
`
import { Component, NgModule } from '@angular/core';
export { NgModule, Component };`
And this is the component I wrote to make dynamic templates possible
`import { Compiler, Input, OnInit, ViewChild, ViewContainerRef } from '@angular/core';
import { Component, NgModule } from<EMAIL_ADDRESS>/*
This Component allows you to compile a template at runtime. Just pass in the template, the needed modules and a context. Make sure that
context is not a complete component, but an object with attributes. Because you would overwrite internal attributes.
e.g. <cu-dynamic [template]="details.html" [context]="{task: task}">
With the example above you could use a template like this {{task.id}}
*/
@Component({
selector: 'cu-dynamic',
template: '<ng-container #vc>'
})
export class DynamicComponent implements OnInit {
@ViewChild('vc', {read: ViewContainerRef}) viewContainer: ViewContainerRef;
@Input() public template: string;
@Input() public modules: any[];
@Input() public context: any;

constructor(private compiler: Compiler) {
}

public static createComponent(compiler: Compiler, template: string, modules?: any[]): any {
    @Component({template})
    class TemplateComponent {
    }

    @NgModule({
        declarations: [TemplateComponent],
        imports: modules
    })
    class TemplateModule {
    }

    const mod = compiler.compileModuleAndAllComponentsSync(TemplateModule);
    const factory = mod.componentFactories.find((comp: any) =>
        comp.componentType === TemplateComponent
    );

    return factory;
}

ngOnInit() {
    const component = DynamicComponent.createComponent(this.compiler, this.template, this.modules);

    const container = this.viewContainer.createComponent(component);
    Object.assign(container.instance, this.context);
    container.changeDetectorRef.detectChanges();
}

}`

That is really all you have to do. It worked for us and we also use forms and ngModel and even the two way binding works. But I hope that we can fix it soon with a compiler-option so, this hack won't be necessary anymore and we can use build-optimizer again. Btw it is also awesome because we don't have to eject the webpack config.

@marcelamsler from the factories you can get VievDef and renderType. After giving them to the JitCompiler

@Alekcei sorry I don't get the context/intention of your comment. Why would I do/need that?

For Some additional info, I also tried to use ngsummary files and pass them to the compiler, like the testcompiler does, it but I was not able to get them.

@p3x-robot how should it be possible to use aot if you don't know the template at compile-time?

We are not going to make any changes to how this works in the short term. The new Ivy view engine (slated to land in 7.0) will allow you to dynamically compose views (note that this is not the same thing as  JIT/ $compile does - allowing full compilation is a security risk, and requires a HUGE amount of slow code to be shipped to browsers).
In short, the output of the compiler will be a public API, so that you could handwrite (or generate at runtime based on whatever) the same thing the compiler spits out. How you arrive at that point will be up to you. See https://github.com/angular/angular/blob/master/packages/core/test/render3/component_spec.ts#L28-L33 for an example of what that looks like. We will not provide you an API to take a string template and compile it at runtime.
See also https://github.com/angular/angular/blob/master/packages/core/src/reflection/reflection_capabilities.ts#L135 - rather than using a decorator, you could use the longhand syntax (this is what the decorator actually does):
import { Component } from '@angular/core';

export class AppComponent  {
  name = 'Angular 5';
}

AppComponent['decorators'] = [{
  type: Component,
  args: [{
    selector: 'my-app',
    template: 'hello'
  }]
}]

This is a private API, is likely to blow up in your face if you rely on it, and is subject to change without warning or mercy, and we will immediately close any issues complaining about it with "I told you so" - again, shipping the JIT compiler to your users is a Bad Idea, and its an especially bad idea to take arbitrary, non-trusted input (eg, "I have random people write templates and drop them into a database, and then fetch them at runtime") and run it through the compiler. You have been warned. JIT is for development ONLY. Please do not rely on this behavior for production applications.

Thanks, @robwormald for taking your time to explain the thoughts of the Angular team on that subject. I do acknowledge many of your points (security, bundle size, performance, ...).
On the other side, I would be very happy if the Angular team could start to honor the need for more dynamic applications. The changes you mentioned for Angular 7 seem to go into that direction. But I am convinced that for web apps with an adaptive UI (e.g. allowing its customers for configuration), a very dynamic solution is required. This could also be a significant differentiator to other web frameworks.

@robwormald
Thank you for your explanation. I think it is a mistake to remove template compilation at runtime. I am with you that everybody should be very careful when they decide to use it, because it has some disadvantages (security, performance). But sometimes you just need it. And to assume that everyone who uses it, is just a noob who knows nothing about security, performance or a proper application architecture, is just wrong. ("I have random people write templates and drop them into a database, and then fetch them at runtime")
In our case we are wrapping an other application, which provides forms, we have to display. So it is just not possible to know the html at compile time. Also we have html content that can be defined and shown whenever the customer wants it. And there we need a lot of html features, which we can not avoid by using structured data from server. But these templates are designed by our own staff and we are also running a security check(whitelist) and optimizations on it, on the server side.
So please consider valid used cases for runtime compilation before removing it from Angular completely.

@Paladinium you have an example of how we could use a module compiled with angular cli, in another application created with angular cli, taking into account that they are not known at compile time

Is there any update of this feature request?
I really wish this feature can be implemented.

I found a viable solution to this problem. can create an angular element
Here is an example:
https://github.com/nrwl/playbook-angular-v6-elements-example
Then insert the script main.js and inline.js at run time and go!
you can create and compile components that can be loaded at run time, this technique does not need the application to know the components at compile time and benefits from the compiler

guys, any updates on it? I faced with this problem during usage of custom decorator. WIth buildOptimizer: true I'm getting all my classes without decorators

No there is no update...
just set buildOptimizer to "false" and reexport the core decorators if you
need them at runtime. Your own decorator should not be removed by normal
build because it is not a core decorator...
As far as I understand the build optimizer, it does not affect size or
performance in a way that would it make really necessary for a normal sized
app.
On Mon, 9 Jul 2018, 10:51 psamusev<EMAIL_ADDRESS>wrote:

guys, any updates on it? I faced with this problem during usage of custom
decorator. WIth buildOptimizer: true I'm getting all my classes without
decorators
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular-cli/issues/9306#issuecomment-403407236,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD5v9DVCAFtHYWHzmNBWeB-j9uCrAP48ks5uExmCgaJpZM4Rl3tJ
.

@marcelamsler From your example above, do I need to know about the @app/aot.decorators?
At the moment during my runtime I'm getting a StaticInjectError:
Uncaught (in promise): Error: StaticInjectorError[n -> n]: StaticInjectorError(Platform: core)[n -> n]: NullInjectorError: No provider for n!
Could this be my DI for the Compiler from @angular/platform-browser-dynamic?

No, I also use the compiler from core directly. Make sure that all Modules for all components, pipes are passed in in "modules"-input. Also make sure, that your custom components you want to use (and their dependencies are using the re-exported decorators. Here is my current code for the exports:
`import { Component, Directive, HostListener, Input, NgModule, Pipe } from '@angular/core';
export { Component, Directive, HostListener, Input, NgModule, Pipe };`

@robwormald As others have expressed, there are companies (like mine) that want to use Angular but have use cases which would be very well served by being able to dynamically compile an html template into a component at runtime, knowing the risks and performance implications.  You may want to reconsider this as a valid use case.

@marcelamsler I'm able to make this work without re-exporting the decorators.  I only have to set build-optimizer=false.  Of course this increases my output main.js file size by 40%.

@lefoulkrod why not use angular element?

where is my angular 1.x ...

Is there an update or work around for this issue yet?

Here are some of my observations relative to this problem (all 3 ways implies aot compilation enabled and combination of lib with custom decorators and project importing it)

the problem is not observed when project imports the lib (with custom decorators) directly (not from node_modules, but from inner folder) and than runs under nginx (not via ng serve --prod, but after it has built with ng build --prod).
the problem is not observed when importing project (importing the lib with custom decorators) disables optimization even if it is started with ng serve --prod
the problem is not observed when importing project compiles to 'es5' target with ng serve --prod
The problem is raised only when these factors meet together: optimization enabled (by default), es2015 target (by default) and webpack server (also by default).

Please can somebody tell me whether solutions 1 and 3 are stables workarounds? Can we rely on this not awaiting the knife in our back? What is the preferable way and why 1 and 3 actually works?
Still very waiting this flag (preserving custom annotations)! Angular team, please pay attention to this, your users are unhappy without it! It's actually a kind of trap: you writing custom decorators and all works well until you develop all the stuff. Next you go live of your project and everyghing wraps into a singularity because angular decides you don't need them as well as they don't and "optimizing" it without any care or choose.
Just have a look at this instruction how to achive this functionality here: https://blog.angularindepth.com/converting-typescript-decorators-into-static-code-using-tsquery-tstemplate-and-transforms-8c65d606a517 It's a kind of Fifty Shades of Grey. Do you think it's normal to rewrite AST tree in 2019 to bring this to project?

I have similar use-case: I need an Angular module which is compiled and stored on a remote server, and loaded into the main app in the runtime. Having in-browser compiler in the AOT build of the main app would solve the issue.
But since there is no solution for this yet, I ended up with a different approach: compiling my remote module as an AOT bundle, using a custom builder script for that. And that is where this great article came handy: https://blog.angularindepth.com/building-extensible-dynamic-pluggable-enterprise-application-with-angular-aed8979faba5

This came up in a CLI meeting recently. @robwormald's previous comment is still a pretty accurate description of our feelings on the matter. Using JIT in a prod application causes a lot of performance and security issues. JIT is intended for development use only, and we don't expect that to change anytime soon.

Here's a link to a relevant discussion, if anybody still needs AOT-compilation of a separate module which is then included to the main application in the runtime via HTTP request: https://github.com/angular/angular/issues/20875#issuecomment-634818485


Portals not using RoomTransitions

I am trying to setup portals in my game to teleport the player ( Not between layers, just to different areas on same layer).
Issue I am running into is that when using the sample code
func _on_body_entered(body: Node2D) -> void:
	# If player entered and isn't doing an event (event in this case is entering the portal).
	if body.is_in_group(&"player") and not body.event:
		body.event = true
		body.velocity = Vector2()
		# Tween the player position into the portal.
		var tween := create_tween()
		tween.tween_property(body, ^"position", position, 0.5).set_ease(Tween.EASE_IN).set_trans(Tween.TRANS_SINE)
		await tween.finished
		# After tween finished, change the map.
		Game.get_singleton().load_room(target_map)
		# A trick to reset player's event variable when it's safe to do so (i.e. after some frames).
		get_tree().create_timer(0.2).timeout.connect(body.set.bind(&"event", false))
		# Delta vector feature again.
		Game.get_singleton().reset_map_starting_coords()

My Player when teleporting is appearing under my tileset and outside my target_map . I believe this is because player position stays the same with the above code, and my portal is at (1884,3335) and my destination's floor is (X,1080). So my player's position is exactly the same when going through the portal. And he falls out of bounds.
You can see this in the following output.
(2090.716, 3466.333) <-- Walking to portal
(2065.716, 3466.333)
(2065.575, 3466.232) <-- Sucked in to portal
(2064.841, 3465.701)
(2063.478, 3464.716)
(2062.224, 3463.81)
(2059.828, 3462.078)
(2056.822, 3459.905)
(2054.484, 3458.216)
(2050.486, 3455.326)
(2045.91, 3452.019)
(2042.546, 3449.587)
(2037.044, 3445.611)
(2031.011, 3441.251)
(2026.704, 3438.138)
(2019.834, 3433.173)
(2012.492, 3427.866)
(2007.349, 3424.149)
(1999.279, 3418.317)
(1990.81, 3412.196)
(1984.956, 3407.965)
(1975.886, 3401.409)
(1966.497, 3394.624)
(1960.077, 3389.984)
(1950.23, 3382.867)
(1940.153, 3375.584)
(1933.325, 3370.649)
(1922.943, 3363.146)
(1912.427, 3355.545)
(1905.358, 3350.437)
(1894.698, 3342.732)
(1884, 3335) <-- Teleport starts
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3335)
(1884, 3339.167) <-- New Map loads in and my player falls
(1884, 3347.5)
(1884, 3360)
(1884, 3376.667)
(1884, 3397.5)
(1884, 3422.5)
(1884, 3451.667)
(1884, 3485)
(1884, 3522.5)

I figure it would be nice to have a way to decide where the player spawns when going through a portal maybe with Spawn_Points (you could argue this should be on the developer not the plugin) or have portals considered when using the RoomTransitions module. I think Portals are a pretty standard metroidvania function that may be worth fleshing out the built in functionality. To at the very least, when creating a portal with the sample code, making sure the player spawns inside the room the portal takes them too.
I have worked around this in my game, by having the portal room always look the same, but others may want something else.
If you feel this is not worth pursuing you can just close the issue ,since there are ways to work around it, no questions asked, thanks!  😊
P.S. If you have any better ideas of how I should be handling teleporting with the portal im all ears!

There is another portal type in DarkStaircase.tscn, which I think does what you want (complete with a "spawn point").


Missing key in AutoScalingGroup resource type

cfn-lint version: (cfn-lint --version)
➜ cfn-lint --version
cfn-lint 0.7.4
Description of issue.
The current linting rules do not account for the NotificationConfiguration key within the AWS::AutoScaling::AutoScalingGroup resource type.
Please provide as much information as possible:

Template linting issues:

Please provide a CloudFormation sample that generated the issue.
(Only relevant parts shown)

  WorkloadAutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      NotificationConfiguration:
        TopicARN: !Ref NotificationTopic
        NotificationTypes:
        - autoscaling:EC2_INSTANCE_LAUNCH
        - autoscaling:EC2_INSTANCE_LAUNCH_ERROR
        - autoscaling:EC2_INSTANCE_TERMINATE
        - autoscaling:EC2_INSTANCE_TERMINATE_ERROR
        - autoscaling:TEST_NOTIFICATION

If present, please add links to the (official) documentation for clarification.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-group.html#cfn-as-group-notificationconfigurations

Hi @agamzn -- I did some digging and NotificationConfiguration has been deprecated in favor of NotificationConfigurations (plural), as documented .  Can you update your template and see if this resolves your issue?

Whoops! Nothing to see here. Good catch!


Add bluewallet Support

https://bluewallet.io
and check their LndHub wrapper:
https://github.com/BlueWallet/LndHub
Could one Blitz host multipüle accounts this way?

Testing the guide on this one: https://github.com/dangeross/guides/blob/master/raspibolt/raspibolt_6B_lndhub.md

got stuck on this already:
admin@openomsRaspiBlitz:~/download/redis-5.0.3 $ sudo systemctl status redis
● redis.service - Redis In-Memory Data Store
   Loaded: loaded (/etc/systemd/system/redis.service; disabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2019-02-17 08:32:53 GMT; 4s ago
  Process: 4961 ExecStop=/usr/local/bin/redis-cli shutdown (code=exited, status=203/EXEC)
  Process: 4959 ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf (code=exited, status=203/EX
 Main PID: 4959 (code=exited, status=203/EXEC)

Feb 17 08:32:53 openomsRaspiBlitz systemd[1]: redis.service: Unit entered failed state.
Feb 17 08:32:53 openomsRaspiBlitz systemd[1]: redis.service: Failed with result 'exit-code'.
Feb 17 08:32:53 openomsRaspiBlitz systemd[1]: redis.service: Service hold-off time over, scheduling res
Feb 17 08:32:53 openomsRaspiBlitz systemd[1]: Stopped Redis In-Memory Data Store.
Feb 17 08:32:53 openomsRaspiBlitz systemd[1]: redis.service: Start request repeated too quickly.
Feb 17 08:32:53 openomsRaspiBlitz systemd[1]: Failed to start Redis In-Memory Data Store.
Feb 17 08:32:53 openomsRaspiBlitz systemd[1]: redis.service: Unit entered failed state.
Feb 17 08:32:53 openomsRaspiBlitz systemd[1]: redis.service: Failed with result 'exit-code'.```

recommend to give this a shot for v1.1 - but when takes too long to delay this feature

Recheck on the guide for v1.1 release: https://twitter.com/bluewalletio/status/1096460638799450113

Also take a look at the pebble wallet while on mobile wallet integration.
https://github.com/IndieSquare/pebblewallet

I am very reluctant to use the Bluewallet until they stop sending the xpubs to blockchain.info. https://twitter.com/richblov8s/status/1108883868399095810?s=19

see https://github.com/rootzoll/raspiblitz/issues/810 for developments

Also take a look at the pebble wallet while on mobile wallet integration.
https://github.com/IndieSquare/pebblewallet

Pebble is now discontinued. Some of the team is working on the Nayuta wallet, which is promising, but not open source yet. I tried and paired with my Blitz successfully.

Looked at BlueWallet again, but I don't feel it being ready for recommendation/implementation:

thankfully instead of blockchain.info the wallet connects to Electrum servers. Possible to use a custom server, but only with TCP (unencrypted) and it is not planned to change anytime soon: https://github.com/BlueWallet/BlueWallet/issues/579

Communication with LND is over clearnet too:
https://github.com/BlueWallet/LndHub/issues/33
Despite the screen suggests a https endpoint all tutorials use TCP.

positive is that logging seems to be not too intruding:
https://github.com/BlueWallet/BlueWallet/blob/master/analytics.js

Just to give an update on these issues (thanks for pointing them out, sometimes we get to overwhelmed on our own little world...)
Since release v5.1.0 all calls to 3rd party APIs were removed and all wallet types are connecting to our Electrum Servers. There may be some code leftovers, but this are not being used, the heavy work is done.
TLS support for Electrum was merged to master on https://github.com/BlueWallet/BlueWallet/commit/7b5db85c9b6443f9f39289dc6217025b79b7c646 and it will be released by default on the next store update.
Regarding the LNDhub connection, it supports TLS and the connection is made over SSH.
Any feedback just let me know. Thanks!

Another update.
https://github.com/BlueWallet/BlueWallet/releases/tag/v5.3.0 introduces TLS for Electrum Personal Server and also for Electrs, we had to disable batching for this specific implementations since they are not following the specifications completely.

@ncoelho Does the LNDhub connection also support TOR?
@openoms With TOR and the above mentioned changes for Electrum, would you then feel it´s ready for implementation on Raspiblitz?

@INSG2020 my concerns above have been fixed, so would be happy to see LNDhub on the RaspiBlitz as an optional service.
The connection for clients would need en exposed port / remote-ssh-tunnel or Tor2IP service.
Not sure if BlueWallet plays well with Orbot to connect to an LNDHub serving on a .onion.
You can already connect BlueWallet to the Electrum Rust Server Running on the Blitz.
Also there is a browser based LN onboarding service in the form of LNbits.

Just to mention the lastest LNbits (as of v1.6.1 RaspiBlitz release) .. has a LNDhub plugin. Maybe that is already a start for some ...

I stated a LndHub installation script in https://github.com/rootzoll/raspiblitz/issues/2320, but had problems setting the correct path for nodejs. If someone wants to continue ...

It looks like that Redis will get a default service for the WebUI BackendAPI with v1.8 .. so I push this to v1.8.1 to then take a look again.

redis.service is working, just having trouble with babel lines that copy files. So this is close to be accomplished if someone debugs further

I installed this recently on RaspiBolt. Is the script above still the latest? I see a few problems. I can't test it without a RaspiBlitz, but here are the things I would change:

Your script runs as user admin, right? Then the git clone command shouldn't need a sudo
The sudo npm install command should be run from the /home/bitcoin/.lndhub directory, but seems it's run from /home/admin instead? It has to move down a few lines in the script.
Besides, we can run sudo -u bitcoin npm install instead of running it as root
The command sudo npm config set prefix '/home/bitcoin/.npm-global' should be run as user bitcoin instead (sudo -u bitcoin [...]) ... otherwise it will append the prefix to file /root/.npmrc instead of to file /home/bitcoin/.npmrc
Babel should be installed via sudo -u bitcoin npm install -g @babel/core @babel/cli (which will install it into the above-defined npm prefix directory: /home/bitcoin/.npm-global/) .... the current script sets admin's $PATH to /home/bitcoin/.npm-global/bin but then installs babel somewhere else, so it won't be found by anyone
Once the above is fixed, we can run sudo -u bitcoin babel ./ --out-dir ./build --copy-files --ignore node_modules without the npm run prefix ... but we need to make sure that bitcoin's $PATH includes /home/bitcoin/.npm-global/bin
Delete this line: sudo node /home/bitcoin/.lndhub/build/index.js ... this launches LndHub and shouldn't be part of the build script

Regarding the port forwarding:
I decided to only expose LndHub over Tor. BlueWallet supports that, you just enter the Onion link in the app.
Thus, one could delete these two lines:
sudo ufw allow 3050 comment 'allow LndHub'
sudo ufw reload

And instead add a Hidden Service in /etc/tor/torrc:
# External port 3000, internal port 3050
HiddenServiceDir /var/lib/tor/hs_lndhub_3000/
HiddenServicePort 3000 <IP_ADDRESS>:3050

Thanks for the corrections. Local IP access needs to be a allowed, a self signed certificate can be used.
You are in the right direction, finish this to claim the bounty.

Okay I can finish it this week. The Blitz environment is close enough to the Bolt, so I should be able to test it on a Bolt with some modifications.

@xanoni , how's it going? i'd love to contribute by adding to the 200k bounty.
it seems like a number of us are trying to execute this "Uncle Jim" BlueWallet/LndHub/LNBits/Raspiblitz plan. i'm not programming savvy, perhaps i should've bought an Umbrel but i am enjoying the struggle at times when things actually work.

@xanoni , how's it going? i'd love to contribute by adding to the 200k bounty.
it seems like a number of us are trying to execute this "Uncle Jim" BlueWallet/LndHub/LNBits/Raspiblitz plan. i'm not programming savvy, perhaps i should've bought an Umbrel but i am enjoying the struggle at times when things actually work.

Good that you're responding here because I wasn't sure if anyone still cared about this, and there's always endless other stuff.
It shouldn't be difficult, I already have it running on RaspiBolt. Just needs to be repackaged and tested.

@xanoni , how's it going? i'd love to contribute by adding to the 200k bounty.
it seems like a number of us are trying to execute this "Uncle Jim" BlueWallet/LndHub/LNBits/Raspiblitz plan. i'm not programming savvy, perhaps i should've bought an Umbrel but i am enjoying the struggle at times when things actually work.

Good that you're responding here because I wasn't sure if anyone still cared about this, and there's always endless other stuff.
It shouldn't be difficult, I already have it running on RaspiBolt. Just needs to be repackaged and tested.

do you have a Blitz to do so?

@xanoni , how's it going? i'd love to contribute by adding to the 200k bounty.

it seems like a number of us are trying to execute this "Uncle Jim" BlueWallet/LndHub/LNBits/Raspiblitz plan. i'm not programming savvy, perhaps i should've bought an Umbrel but i am enjoying the struggle at times when things actually work.

Good that you're responding here because I wasn't sure if anyone still cared about this, and there's always endless other stuff.

It shouldn't be difficult, I already have it running on RaspiBolt. Just needs to be repackaged and tested.

do you have a Blitz to do so?

Not right now but I know well enough how the Blitz works so I think I can fake one on the Bolt.
Someone might have to do a proper test for me, though :)

@xanoni , how's it going? i'd love to contribute by adding to the 200k bounty.

it seems like a number of us are trying to execute this "Uncle Jim" BlueWallet/LndHub/LNBits/Raspiblitz plan. i'm not programming savvy, perhaps i should've bought an Umbrel but i am enjoying the struggle at times when things actually work.

Good that you're responding here because I wasn't sure if anyone still cared about this, and there's always endless other stuff.

It shouldn't be difficult, I already have it running on RaspiBolt. Just needs to be repackaged and tested.

do you have a Blitz to do so?

Not right now but I know well enough how the Blitz works so I think I can fake one on the Bolt.
Someone might have to do a proper test for me, though :)

i'll volunteer!

@xanoni , how's it going? i'd love to contribute by adding to the 200k bounty.

it seems like a number of us are trying to execute this "Uncle Jim" BlueWallet/LndHub/LNBits/Raspiblitz plan. i'm not programming savvy, perhaps i should've bought an Umbrel but i am enjoying the struggle at times when things actually work.

Good that you're responding here because I wasn't sure if anyone still cared about this, and there's always endless other stuff.

It shouldn't be difficult, I already have it running on RaspiBolt. Just needs to be repackaged and tested.

do you have a Blitz to do so?

Not right now but I know well enough how the Blitz works so I think I can fake one on the Bolt.

Someone might have to do a proper test for me, though :)

i'll volunteer!

Thank you that makes it a lot easier ... my main fear was missing something and frustrating people if there's a bug. I'll try to message you somewhere.

@xanoni , how's it going? i'd love to contribute by adding to the 200k bounty.

it seems like a number of us are trying to execute this "Uncle Jim" BlueWallet/LndHub/LNBits/Raspiblitz plan. i'm not programming savvy, perhaps i should've bought an Umbrel but i am enjoying the struggle at times when things actually work.

Good that you're responding here because I wasn't sure if anyone still cared about this, and there's always endless other stuff.

It shouldn't be difficult, I already have it running on RaspiBolt. Just needs to be repackaged and tested.

do you have a Blitz to do so?

Not right now but I know well enough how the Blitz works so I think I can fake one on the Bolt.

Someone might have to do a proper test for me, though :)

i'll volunteer!

Thank you that makes it a lot easier ... my main fear was missing something and frustrating people if there's a bug. I'll try to message you somewhere.

@jcmillionaire on Reddit.

FYI: So I've started putting this together, but ran out of time and need to finish it later ... don't use this in the current form, please. (It's quite close, though. Stay tuned.)
https://github.com/xanoni/raspiblitz/blob/ab22f59399b5e2a554033d2230a7cae87737897f/home.admin/config.scripts/bonus.lndhub.sh
Diff: https://github.com/xanoni/raspiblitz/commit/ab22f59399b5e2a554033d2230a7cae87737897f

Okay! thank you I have what I need then

So what's the status here? Do you need help @xanoni ?
My linux skills are not enough to really help, but I can test it on my Raspiblitz if you want.

Sorry but I'm currently somewhat disconnected from this world here. Life.

@rootzoll is there some support required on this item? I'd be interested to do some work on this.

@seinzfiction thank you for the offer. Before working to integrate LNDhub itself check if the the latest LNbits might provide an LNDHub plugin which would make it available for the LND and CLN funding source as well.

@rootzoll The current LNBits offers a LNDHub extension already. But as far as I understood and experienced it, this extension offers LNDHub for the LNBits users wallets. Via this way it is not possible to offer LNDHub access for the Raspiblitz Lightning wallet directly.

So multiple users must be handled by LNbits then - they need to create a wallet and get the LNDhub link individually as I undestand
This is indeed less convenient then just having one LNDhub link to share so might justify the standalone LNDhub implementation.

@openoms you're right. LNBits provides custodial Lightning wallets to 3rd party users and the LNDHub extension in LNBits can be used to access those wallets through Blue Wallet directly.

There is a very promising reimplementation of LndHub in Go language here: https://github.com/getAlby/lndhub.go
I recommend to use this as a LndHub/BlueWallet server.

Tried LNbits with LNDHub but Raspiblitz does not save the settings. It loses the wallet and the LNDHub installation. When I create a wallet, and close LNBits, and re-open it again, the wallet is gone.

Tried LNbits with LNDHub but Raspiblitz does not save the settings. It loses the wallet and the LNDHub installation. When I create a wallet, and close LNBits, and re-open it again, the wallet is gone.

@callebtc can you take a look or patch us to the right LNbits person what persistence error this could be with LNDhub plugin?

Tried LNbits with LNDHub but Raspiblitz does not save the settings. It loses the wallet and the LNDHub installation. When I create a wallet, and close LNBits, and re-open it again, the wallet is gone.

I had to bookmark my LNBits on Tor Browser

If the wallet is gone (not the extension) it would rather sound like a deeper db persistance problem. But then nothing in LNbits would work.
@Smiggel what do you mean exactly by "the wallet is gone"?
@rootzoll how do I test this without a blitz?

@rootzoll how do I test this without a blitz?

Maybe @arcbtc can sponsor a RaspiBlitz from FULMO for future testing?

@Smiggel what do you mean exactly by "the wallet is gone"?

Perhaps I am understanding incorrectly how it should, but here is what I do:

Installed LND bits.
Installed LND Hub
Create a wallet
Close the browser
Open the browser and browse to LND bits

After point 5 I expect to see my wallet from step 3. However, I don’t see the wallet anymore. When I look at the extensions, I see the “install” button again for LND hub.
I would except the wallet to be there and the extension installed.

I would except the wallet to be there and the extension installed.

Did you open the wallet by its URL or just open LNbits ? You either need to save the URL of the wallet or import it to e.g. BlueWallet by using the "Export-via-QR-to-phone" feature, see the popup you click when generating a wallet 😋 Sats are still on your node so nevermind about the lost link.

Thanks, I will give that a try.

BlueWallet doesn't support Lightning anymore
@rootzoll closing this for now


Borders and BG fix

Also makes Ascension button turn green if auto enabled

This is stupid, it wont allow me any changes unless I merge my fork, and it will show it here for no reason after I do


Introduce PoP phone number field to application

Context
We want to record the PoP's (person on probation) phone number in the
contact details section of the application.
This is an optional field and should not produce errors if it is not supplied.
Changes in this PR
Screenshots of UI changes

Release checklist
Release process documentation.
Pre-merge checklist

[ ] Are any changes required to dependent services for this change to work?
(eg. CAS API)
[ ] Have they been released to production already?

Post-merge checklist

[ ] Manually approve
release to test
[ ] Manually approve
release to preprod
[ ] Manually approve
release to prod

 Add a phone number for PoP

Oh I hadn't spotted that this is a new page (rather than merging into an existing page) so it should appear as a new incomplete question out of the box right? Meaning any in progress referral can easily see what they need to go back and add?

As it's an optional field I believe in-progress applications will be free to submit without completing this question. This is because we check for any page errors just before we submit.
As it's optional, we don't include an error if it's not supplied.

@tahb
Actually, you're right, it will require you to go back and complete the section. The UX is a little iffy as it looks like you've lost progress, but I've confirmed it does work.
https://github.com/ministryofjustice/hmpps-temporary-accommodation-ui/assets/6377078/355d775d-73a4-4b51-980b-cabee4f68bf1


2.8.2400 Sudden high Ram then dump gone

This morning redis suddenly had 50GB ofRAM (maxmemroy is only set to 10000mb and maxheab to 30000mb)
As this slowed down the whole server . then I restarted redis, as a result the dump file was gone !
There were no errors in redis logs.
Any explenations  on this?
Can the redis dump get lost if the service restart is triggerd while redis is saving?
Is there a way to configure redis du have a backup dump is the current one "gets lost" ?
Any advice what to do to avoid this?

now dump gone one more time :-/ I don't understand why t his happens :-/

@Gentlehag
the RDB rewrite process is:

create a new temp file where the new RDB is first saved
if the temp file has been saved successfully close the original RDB and rename the temp file to the new file.

Give the previously described algorithm, killing the Redis process is a safe thing by the RDB point of view, even if the killing occurs while Redis is saving to disk. The only problem that may happen is that the temp file is left behind along with the original RDB.
The problem you described is new to me since I've never heard of such a problem.
Are you sure that the RDB file is not deleted externally (i.e. antivirus or any other third party software)?

yes i'm sure that it is not deleted. this time i didn't event restart redis.

@Gentlehag
everytime you shutdown Redis the RDB file is deleted?

nope can't tell when.
The problem half an hour ago happend in the running evironment without any administrative interaction with redis

@Gentlehag
Redis doesn't have any code that executes a "delete", it only renames the RDB and given the renaming is done by Windows it's an atomic operation that doesn't and up deleting the original file and the destination.
Is there any temp-dump.rdb file left over?

no there was no empty tmp-dump.rdp
what would happen if the target is locked?

@Gentlehag
if the target is locked the Windows rename command would fail leaving both the RDB and the temp file behind.
There isn't a command to explicitly delete the RDB file, there is a command to remove all the keys from the in-memory data-set that in turn may cause the RDB file to be empty, but not to delete it.
If both the RDB and the temp files are gone, I suspect something external.

to be exactly i don't know wether it is gone or just completly empty.
Is way to log if that command occurs? or even better to prevent this command? we have no use case for it.

@Gentlehag
you may rename or disable the FLUSHALL command using the general command-disable mechanism as described in the config file.

@Gentlehag
btw, last time we had a reported issue similar to yours, it was a compromised server, make sure it's not your case.

can you provide me an example for the command-disable ?
in my confdig the command disbale isn't described.

i found a possible issue with one redis instance which may led to that someone could have sent commands (like flushall) to this instance.
After solving this issue and also blocking critical commands it didn't happened again. so I close this case.


_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
shelldocs
0m  0s
Shelldocs was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 11s
Maven dependency ordering for branch

+1 :green_heart:
spotless
0m 42s
branch has no errors when running spotless:check.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  4s
Maven dependency ordering for patch

+1 :green_heart:
shellcheck
0m  0s
There were no new shellcheck issues.

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
spotless
0m 39s
patch has no errors when running spotless:check.

_ Other Tests _

+0 :ok:
asflicense
0m  0s
ASF License check generated no output?

2m 56s

Subsystem
Report/Notes

Docker
ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5646/1/artifact/yetus-general-check/output/Dockerfile

GITHUB PR
https://github.com/apache/hbase/pull/5646

Optional Tests
dupname asflicense spotless shellcheck shelldocs

uname
Linux 5754a981a4dc 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/hbase-personality.sh

git revision
master / 3a91c5b698

Max. process+thread count
45 (vs. ulimit of 30000)

modules
C:  U:

Console output
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5646/1/console

versions
git=2.34.1 maven=3.8.6 shellcheck=0.8.0

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


Add case study for bzip2

Signed-off-by: Steven Wirsz<EMAIL_ADDRESS>

Nice work @swirsz -- I wonder what's missing in order to achieve 100% reachability of the project, which would be neat.
Looking at one of the recent reports in the "Project functions overview" table, and sorting by the rightmost column, "Undiscovered complexity" I get:

It looks like the BZ2 family of functions could be hit (https://storage.googleapis.com/oss-fuzz-coverage/bzip2/reports/20221020/linux/src/bzip2/bzlib.c.html#L1472). It would be neat if you made a fuzzer for those targets as that would get reachability to 100% -- the first of it's kind in the fuzz introspector world I believe! Coverage will probably stay a bit lower because of e.g. error-handling code (we could deploy some mechanism that forces some of the routines that trigger error handling to fail in order to get 100% coverage, e.g. force malloc to fail once in a while).
Note the picture also shows a bug -- some missing coverage interpretation for LLVMFuzzerTestOneInput and some third-party functions!

@DavidKorczynski Very interesting challenge. I submitted another PR to try to reach that 100%.  FI reports 95% coverage (85 out of 89) and 99% CC coverage.

Note that BZ2_bzdopen is fully utilized by one of the fuzz drivers.

That's very cool indeed @swirsz -- I'll see if I can sort these false positives in terms of coverage soon -- at least LLVMFuzzerTestOneInput should be covered. Do you know if the fprintf and __assert_fail are indeed reached?

There is no __assert_fail function, and FI's links to both of those references are invalid. There is a BZ2_bz__AssertH__fail function.
I tried to make an arbitrary call to BZ2_bz__AssertH__fail and make an arbitrary fprintf statement (shows up green in FI's call tree) but the coverage still shows 85/89 functions covered and fprintf uncalled. (Same coverage as before manually adding these 2 statements)

Thanks for the details @swirsz -- I'll look at this in a few days and then I'll sort this out in terms of ensuring the functions that are not there should not be in the table.

@swirsz I think I got this one identified/solved.
It's because of https://github.com/ossf/fuzz-introspector/blob/a47cb284d1f65cd51db809e46d96179a26497bb9/src/fuzz_introspector/constants.py#L44
which is used here:
https://github.com/ossf/fuzz-introspector/blob/737ba7226934acdc724f4d81daa1bcb9f5b14be6/src/fuzz_introspector/utils.py#L247-L257
and thus blocks the functions in the list, which are used when finding "functions reached" for a given function:
https://github.com/ossf/fuzz-introspector/blob/737ba7226934acdc724f4d81daa1bcb9f5b14be6/src/fuzz_introspector/datatypes/function_profile.py#L47
Uncommenting the assert and printf logic from the banned list, I get:

@swirsz after unblocking these things, bzip2 gets:

https://github.com/ossf/fuzz-introspector/pull/577 fixes this!

@swirsz @Navidem 100% reachability here: https://storage.googleapis.com/oss-fuzz-introspector/bzip2/inspector-report/20221103/fuzz_report.html
Does it make sense to add this to the doc and then let's get this merged?

https://storage.googleapis.com/oss-fuzz-introspector/bzip2/inspector-report/20221103/fuzz_report.html

This is amazing! Thanks @swirsz and @DavidKorczynski.

Does it make sense to add this to the doc and then let's get this merged?

Definitely we should use the latest report, but AFAIK Steven wanted to update the posting text with more details or something. So I'd wait for him to make another push.

@swirsz we just need changes in doc/CaseStudies.md.

cleaned up the PR

Thanks!