TypeError: Object of type mappingproxy is not JSON serializable

Running the following command "python mce_cli.py produce -d bootstrap_mce.dat" produces this error "TypeError: Object of type mappingproxy is not JSON serializable".  Do I need to make any changes in encoder.py or mce_cli.py files.

$ python mce_cli.py produce -d bootstrap_mce.dat
Producing MetadataChangeEvent records to topic MetadataChangeEvent. ^c to exit.
Traceback (most recent call last):
File "mce_cli.py", line 112, in
main(parser.parse_args())
File "mce_cli.py", line 94, in main
produce(conf, args.data_file)
File "mce_cli.py", line 36, in produce
producer.produce(topic=topic, value=content)
File "/Users/pshaji/.local/lib/python3.7/site-packages/confluent_kafka/avro/init.py", line 80, in produce
value = self._serializer.encode_record_with_schema(topic, value_schema, value)
File "/Users/pshaji/.local/lib/python3.7/site-packages/confluent_kafka/avro/serializer/message_serializer.py", line 105, in encode_record_with_schema
schema_id = self.registry_client.register(subject, schema)
File "/Users/pshaji/.local/lib/python3.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 216, in register
body = {'schema': json.dumps(avro_schema.to_json())}
File "/opt/anaconda3/lib/python3.7/json/init.py", line 231, in dumps
return _default_encoder.encode(obj)
File "/opt/anaconda3/lib/python3.7/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/opt/anaconda3/lib/python3.7/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/opt/anaconda3/lib/python3.7/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type mappingproxy is not JSON serializable

Unable to resolve this issue.

Hi @PriyaShaji, have you tried to run the mce-cli.py in virtualenv with Python 2.7 above?
All the dependencies should be included in the requirements.txt, no extra changes should be made.
Meanwhile, you may also leverage ingestion docker for the pipeline as we introduced recently.

I tried to create and activate virtualenv in the mce-cli folder which is in the metadata-ingestion. Then when I run the pip install --user -r requirements.txt , I get the following error:
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
(mypython) (base) aplc02sw05gg8wl:mypython pshaji$ pip install --user -r requirements.txt
ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
Python version: Python 3.7.4

Hi @PriyaShaji:
First, the --user working when --system-site-packages were given to virtualenv. If you are going to leverage virtualenv, please remove the --user flag after activation.
Second, the error message indicated the 'confluent-kafka[avro]==1.1.0' not installed which is located in the requirements.txt. pip install -r requirements.txt if you are in virtualenv
Last, all the scripts were verified with Python 2.7 above, strongly recommend to play with Python 2.

Thank you, It's working now, except when I run python mce_cli.py produce -d bootstrap_mce.dat , It shows: confluent_kafka.avro.error.ClientError: Unable to register schema. Error code:404

$ python mce_cli.py produce -d bootstrap_mce.dat
Producing MetadataChangeEvent records to topic MetadataChangeEvent. ^c to exit.
Traceback (most recent call last):
File "mce_cli.py", line 108, in
main(parser.parse_args())
File "mce_cli.py", line 88, in main
produce(conf, args.data_file, args.schema_record)
File "mce_cli.py", line 32, in produce
producer.produce(topic=topic, value=content)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/init.py", line 80, in produce
value = self._serializer.encode_record_with_schema(topic, value_schema, value)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/serializer/message_serializer.py", line 105, in encode_record_with_schema
schema_id = self.registry_client.register(subject, schema)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 225, in register
raise ClientError("Unable to register schema. Error code:" + str(code))
confluent_kafka.avro.error.ClientError: Unable to register schema. Error code:404

I didn't change the schema-registry port in mce-cli.py, hence the issue is solved. However, python mce_cli.py produce -d bootstrap_mce.dat  shows requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine('No status line received - the server has closed the connection',))

$ python mce_cli.py produce -d bootstrap_mce.dat
Producing MetadataChangeEvent records to topic MetadataChangeEvent. ^c to exit.
Traceback (most recent call last):
File "mce_cli.py", line 108, in
main(parser.parse_args())
File "mce_cli.py", line 88, in main
produce(conf, args.data_file, args.schema_record)
File "mce_cli.py", line 32, in produce
producer.produce(topic=topic, value=content)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/init.py", line 80, in produce
value = self._serializer.encode_record_with_schema(topic, value_schema, value)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/serializer/message_serializer.py", line 105, in encode_record_with_schema
schema_id = self.registry_client.register(subject, schema)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 217, in register
result, code = self._send_request(url, method='POST', body=body)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 166, in _send_request
response = self._session.request(method, url, headers=_headers, json=body)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine('No status line received - the server has closed the connection',))

I tried disabling proxy network, but it did'nt work out. still shows the connection error. I am using chrome as a default browser and python 2.7. Any insights on how to resolve this issue?

I changed http to https in the mce_cli.py file but no success, it shows:
requests.exceptions.ConnectionError: ('Connection aborted.', error(0, 'Error'))

(NAME) aplc02sw05gg8wl:mce-cli pshaji$ python mce_cli.py produce -d bootstrap_mce.dat
Producing MetadataChangeEvent records to topic MetadataChangeEvent. ^c to exit.
Traceback (most recent call last):
File "mce_cli.py", line 108, in
main(parser.parse_args())
File "mce_cli.py", line 88, in main
produce(conf, args.data_file, args.schema_record)
File "mce_cli.py", line 32, in produce
producer.produce(topic=topic, value=content)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/init.py", line 80, in produce
value = self._serializer.encode_record_with_schema(topic, value_schema, value)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/serializer/message_serializer.py", line 105, in encode_record_with_schema
schema_id = self.registry_client.register(subject, schema)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 217, in register
result, code = self._send_request(url, method='POST', body=body)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 166, in _send_request
response = self._session.request(method, url, headers=_headers, json=body)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(0, 'Error'))
(NAME) aplc02sw05gg8wl:mce-cli pshaji$

Hi @PriyaShaji,
According to the log, it looks like the mce-cli.py cannot talk to the Kafka broker, would you be able to run the docker stats to confirm all the prerequisite components are up as described in the quickstart guide?

Yes, Kafka is not running,

docker logs kafka-setup gives the following:

[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version: 5.3.0-ccs
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId: 29b8bc3c33c75c2a
[main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka startTimeMs:<PHONE_NUMBER>359
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
[kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.
(NAME) aplc02sw05gg8wl:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \broker
true
(NAME) aplc02sw05gg8wl:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \kafka-setup
false
(NAME) aplc02sw05gg8wl:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \elasticsearch-setup
false
(NAME) aplc02sw05gg8wl:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \zookeeper
true
(NAME) aplc02sw05gg8wl:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \schema-registry
true
(NAME) aplc02sw05gg8wl:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \schema-registry-ui
true
(NAME) aplc02sw05gg8wl:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \zookeeper
true
(NAME) aplc02sw05gg8wl:mce-cli pshaji$

I also wanted to see of all the prerequisite containers are working . kafka-setup and elasticsearch-setup are not running

In docker-compose up --build : kafka-setup gives this error:

kafka-setup             | [kafka-admin-client-thread | adminclient-1] WARN org.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-1] Connection to node -1 (broker/<IP_ADDRESS>:29092) could not be established. Broker may not be available.

SO, initially when i run docker-compose up --build , it gave me this error:

ERROR: for schema-registry  Cannot start service schema-registry: driver failed programming external connectivity on endpoint schema-registry (2c3ba2dfd1306de05f46352562a06ccf55fd55246021fe9fa161cd7955743256): Error starting userland proxy: listen tcp <IP_ADDRESS>:8081: bind: address already in use

So, to resolve that I changed the schema-registry port address to 8082, then it did'nt show that error. was that an appropriate resolve?

when i run docker-compose up --build: for broker I get the following:

broker                  | [2019-12-26 16:36:35,221] DEBUG [Controller id=1] Topics not in preferred replica for broker 1 Map() (kafka.controller.KafkaController)
broker                  | [2019-12-26 16:36:35,221] TRACE [Controller id=1] Leader imbalance ratio for broker 1 is 0.0 (kafka.controller.KafkaController)
broker                  | [2019-12-26 16:41:30,126] INFO [GroupMetadataManager brokerId=1] Removed 0 expired offsets in 6 milliseconds. (kafka.coordinator.group.GroupMetadataManager)

for elasticsearch-setup:

elasticsearch-setup     | 2019/12/26 16:31:13 Problem with request: Get http://elasticsearch:9200: dial tcp <IP_ADDRESS>:9200: connect: connection refused. Sleeping 1s

Not able to resolve this issue.

Hi @PriyaShaji ,
Please be aware of the quickstart docker image orchestrating all the components to deploy the datahub full-stack, that is to say, the component may have dependencies on each other while startup the system, the details can be found in docker-compose.yml.
As for the log in the kafka-setup is due to the Kafka broker not brought up, causing the topics are unable to be created.
On the elasticsearch-setup side, the log illustrates the process for the pre-create search indices, which continues dialing to the elastic search host until handshaking.
Regarding your bootstrapping purpose, the first step is to make sure your 9092 port for Kafka broker not occupied and can be brought up successfully.

@PriyaShaji
Use sudo lsof -i TCP:9092  to find the Process occupying 9092.
You can then pipe it to kill command.  kill -p {PID}
https://stackoverflow.com/questions/3855127/find-and-kill-process-locking-port-3000-on-mac
https://linux.die.net/man/1/kill

@RealChrisL The broker is using two port
- "29092:29092"
- "9092:9092"
PLAINTEXT://broker:29092 , kafka wont communicate to this ?

Hi @PriyaShaji
Were you able to solve the issue? Do you still need help?

Would the error showing because its not able to detect the dat file. As in mac, the dat file has the vlc icon. Wondering if the error is related to that?

I was trying to see which line in the mce_cli.py is not running or where the program is stopping:
producer.produce(topic=topic, value=content) is not being executed or the point in the program where it throws error. I modified the .py file to see till how far it's working properly. I have attacjed the python file too. Below is the output for the `python mce_cli.py produce -d bootstrap_mce.dat
`

(NAME) t5ftsa06614:mce-cli pshaji$ python mce_cli.py produce -d bootstrap_mce.dat
Producing MetadataChangeEvent records to topic MetadataChangeEvent. ^c to exit.
success1!
success2!
success3!
success4!
success6!
{'auditHeader': None, 'proposedSnapshot': ('com.linkedin.pegasus2avro.metadata.snapshot.CorpUserSnapshot', {'urn': 'urn:li:corpuser:datahub', 'aspects': [{'active': True, 'fullName': 'Data Hub', 'displayName': 'Data Hub', 'email'<EMAIL_ADDRESS>{}]}), 'proposedDelta': None}
success7!
MetadataChangeEvent
Traceback (most recent call last):
File "mce_cli.py", line 118, in
main(parser.parse_args())
File "mce_cli.py", line 98, in main
produce(conf, args.data_file, args.schema_record)
File "mce_cli.py", line 41, in produce
producer.produce(topic=topic, value=content)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/init.py", line 80, in produce
value = self._serializer.encode_record_with_schema(topic, value_schema, value)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/serializer/message_serializer.py", line 105, in encode_record_with_schema
schema_id = self.registry_client.register(subject, schema)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 217, in register
result, code = self._send_request(url, method='POST', body=body)
File "/Users/pshaji/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 166, in _send_request
response = self._session.request(method, url, headers=_headers, json=body)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/Users/pshaji/.local/lib/python2.7/site-packages/requests/adapters.py", line 498, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine('No status line received - the server has closed the connection',))

Any help would be appreciated. Thank you in advance!

@RealChrisL could you help @PriyaShaji debug the issue?

Hi, @PriyaShaji, thanks for the information, looks like the issue looping back to Dec 19, 2019.
The BadStatusLine connection error is caused while producing the sample event to the Kafka, are you able to run the docker stats to confirm all the prerequisite components are satisfied in the quickstart guide before executing the script?
Thanks.

Thank you for the reply @RealChrisL @mars-lan
I checked docker stats, all containers are running , just the elasticsearch-setup and kafka setup have stopped. The reason was explained by @keremsahin1 https://github.com/linkedin/datahub/issues/1516#issuecomment-579014829
There is also a docker command which can be used to see if a container is running or not:
docker inspect -f '{{.State.Running}}' $container_name
I have inspected via this command too, and all containers are running.

(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \mysql
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \zookeeper
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \broker
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \kafka-setup
false
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \schema-registry
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \schema-registry-ui
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \elasticsearch
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \kibana
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \elasticsearch-setup
false
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \datahub-gms
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \datahub-frontend
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \datahub-mae-consumer
true
(NAME) t5ftsa06614:mce-cli pshaji$ docker inspect -f '{{.State.Running}}' \datahub-mce-consumer
true
(NAME) t5ftsa06614:mce-cli pshaji$

while building the docker compose containers docker-compose up --build I got the warning for either mentioning ssl certificate or disabling it. So, in my docker-compose.yml file , I added verifyServerCertificate=false&useSSL=true , so it did not show that warning again. EBEAN_DATASOURCE_URL=jdbc:mysql://mysql:3306/datahub?verifyServerCertificate=false&useSSL=true.

datahub-gms:
image: keremsahin/datahub-gms:latest
hostname: datahub-gms
container_name: datahub-gms
ports:
- "8080:8080"
environment:
- EBEAN_DATASOURCE_USERNAME=datahub
- EBEAN_DATASOURCE_PASSWORD=datahub
- EBEAN_DATASOURCE_HOST=mysql:3306
- EBEAN_DATASOURCE_URL=jdbc:mysql://mysql:3306/datahub?verifyServerCertificate=false&useSSL=true
- EBEAN_DATASOURCE_DRIVER=com.mysql.jdbc.Driver
- KAFKA_BOOTSTRAP_SERVER=broker:29092
- KAFKA_SCHEMAREGISTRY_URL=http://schema-registry:8083
- ELASTICSEARCH_HOST=elasticsearch
- ELASTICSEARCH_PORT=9200
- NEO4J_URI=bolt://neo4j
- NEO4J_USERNAME=neo4j
- NEO4J_PASSWORD=datahub

Wondering if this is somewhere related to the error I am receiving while bootstraping the data.

Using or not using ssl certificate does'nt makes a difference, I tried building without EBEAN_DATASOURCE_URL=jdbc:mysql://mysql:3306/datahub?verifyServerCertificate=false&useSSL=true, all the containers are running, but it still shows error while bootstrapping. Unable to resolve the bootstrap issue so far.

@PriyaShaji ,  in your case, may relate to the connection issue between the Kafka broker and producer.
Recently, we introduced the standalone ingestion docker image to provide a clean environment for bootstrapping, would you please give a try by following the steps on the README, thanks!

Thank you @RealChrisL . Does the docker build -t ingestion -f docker/ingestion/Dockerfile . takes long to execute?

No worries, it worked second time.

It's not working either, while running docker-compose up, it gives the following:

(NAME) APLC02SW05GG8WL:ingestion pshaji$ docker-compose up
Recreating ingestion ... done
Attaching to ingestion
ingestion    | Traceback (most recent call last):
ingestion    |   File "mce_cli.py", line 108, in
ingestion    |     main(parser.parse_args())
ingestion    |   File "mce_cli.py", line 88, in main
ingestion    | Producing MetadataChangeEvent records to topic MetadataChangeEvent. ^c to exit.
ingestion    |     produce(conf, args.data_file, args.schema_record)
ingestion    |   File "mce_cli.py", line 32, in produce
ingestion    |     producer.produce(topic=topic, value=content)
ingestion    |   File "/root/.local/lib/python2.7/site-packages/confluent_kafka/avro/init.py", line 80, in produce
ingestion    |     value = self._serializer.encode_record_with_schema(topic, value_schema, value)
ingestion    |   File "/root/.local/lib/python2.7/site-packages/confluent_kafka/avro/serializer/message_serializer.py", line 105, in encode_record_with_schema
ingestion    |     schema_id = self.registry_client.register(subject, schema)
ingestion    |   File "/root/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 217, in register
ingestion    |     result, code = self._send_request(url, method='POST', body=body)
ingestion    |   File "/root/.local/lib/python2.7/site-packages/confluent_kafka/avro/cached_schema_registry_client.py", line 166, in _send_request
ingestion    |     response = self._session.request(method, url, headers=_headers, json=body)
ingestion    |   File "/root/.local/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
ingestion    |     resp = self.send(prep, **send_kwargs)
ingestion    |   File "/root/.local/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
ingestion    |     r = adapter.send(request, **kwargs)
ingestion    |   File "/root/.local/lib/python2.7/site-packages/requests/adapters.py", line 516, in send
ingestion    |     raise ConnectionError(e, request=request)
ingestion    | requests.exceptions.ConnectionError: HTTPConnectionPool(host='schema-registry', port=8083): Max retries exceeded with url: /subjects/MetadataChangeEvent-value/versions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3e4026e710>: Failed to establish a new connection: [Errno 111] Connection refused',))
ingestion exited with code 1

Hi @RealChrisL , the ingestion data is working now, containers are all up and running, I rechecked using the debug files too. When I try to log in to the datahub portal http://localhost:9001  , It loops me back to the login page. Any suggestions?

Hi, @PriyaShaji
Congratulate to move on to the next steps!
By default, we use datahub as both username and password in the login page which lays on the users-gms.
Are you able to see the user info from the MySQL?
If so, would you please provide the stack trace from the containers?

Did you mean to check docker logs for mysql?

You can access the MySQL container and as well as provide the stack trace from the dockers and the front end log while login.

@PriyaShaji Log into MySQL and look to see if you have a metadata_aspect table with users in it. Adding commands and trouble-shooting info below in case it is helpful to you or a future user.
To connect to MySQL:
docker exec  -it mysql mysql -udatahub -pdatahub datahub
Query for users:
select * from datahub.metadata_aspect where aspect = 'com.linkedin.identity.CorpUserInfo' AND urn = 'urn:li:corpuser:datahub';
If you get an error saying the table metadata_aspect cannot be found, run init.sql found in the docker/mysql directory. You can copy the text from the file and paste it onto the MySQL command line.
If you do not get an error, but no records are returned, run the insert command from that same init.sql file.
After that I believe you should be able to bootstrap the loader.

Thankyou @RealChrisL @keremsahin1 @mars-lan @ben5448 , finally, after resolving all issues , sample datasets were loaded and I was able to sign in successfully.  The previous issue I guess was because the path were script chardetect and fastavro was installed was not added on PATH.  After resolving that and other issues thrown before this , datahub was loading without any hinderance.

many more features of datahub to be explored further:), Thankyou!

Thanks for your interest  😄
Close the issue now, feel free to open if having other concerns.

Thanks for your patience and dedication to solve all the issues @PriyaShaji  :)
You definitely spent good amount for debugging and have good learnings. Please feel free to send a PR to fix or improve any documentation that would help others along the way.

Sure! Thank you for making it open source:), it's a great way to learn for interns like me.


Added gmail notification handler

Hi,
I added a gmail handler and I'll open a PR to the stable/helm repo to make use of the same parameters.
I also ran all the shellcheck.net rejections and fixed basic bash stuff.
Hope you'll like it.

@egeland @mumoshu Can you guys check this out?

@omerxx Hey! Thanks for the PR!
I don't have an immediate test environment for this feature, so I appreciate your help. Did you get a chance to verify this to work in a real cluster?

Hi @mumoshu, yes I've created it for my company and we're now testing in production.
I also have changes ready for helm/stable which we use to install this and use the gmail handler. I'll create a PR form them once this one is merged.
Thanks!

Ah, now merging is blocked due to unsigned commits. Would you mind signing according to https://help.github.com/articles/about-gpg/ ?

@mumoshu I will, and also checking the questions by @Multiply

@mumoshu Squashed and signed my commits.
About @Multiply I had another thinking and I actually find it ok. Any problem you see?

@omerxx The output of the echo command is wrong, and is missing the hint about which variable is missing. Have you seen my comment above, with an explanation of what happens?

@mumoshu Fixed @Multiply comment too.

@mumoshu Can we merge?

@omerxx @Multiply Thanks for a lot for your supports ☺

Nice feature addition. 👍
Would have preferred to see the linting stuff (shellcheck) in a separate PR to a new feature - please keep in mind for next time. 😁

@egeland Understood! Thanks!


Bring back InfiniteSum{ITensor} for 2-local Hamiltonians

I've gotten reports from Bo Xiao of CCQ that there appears to be a performance/memory regression in the VUMPS/TDVP code, likely caused by the rewrite of the code for non-local interactions.
To help debug the performance issue, this PR brings back the simpler version of VUMPS/TDVP that was written for 2-local Hamiltonians in terms of sums of ITensors.
The interface is that if you input a InfiniteSum{MPO} it calls the newer version for non-local Hamiltonians (the version that appears to have a performance/memory regression) while inputting an InfiniteSum{ITensor} calls the old version that is limited to 2-local Hamiltonians.
@LHerviou have you noticed any performance issues since your rewrite of the VUMPS/TDVP code that extended it to non-local Hamiltonians?

I'm merging so we can test it out.

I just did timings of the Hubbard model example, with these parameters:
maxdim = 30 # Maximum bond dimension
cutoff = 1e-6 # Singular value cutoff when increasing the bond dimension
max_vumps_iters = 200 # Maximum number of iterations of the VUMPS algorithm at each bond dimension
outer_iters = 5 # Number of times to increase the bond dimension
localham_type = MPO # or ITensor

model_params = (t=1.0, U=10.0, V=0.0)

The timing results I get for the old VUMPS implementation (InfiniteSum{ITensor}) are:
 21.846960 seconds (269.77 M allocations: 25.508 GiB, 18.09% gc time, 0.02% compilation time)

while for the new VUMPS implementation (InfiniteSum{MPO}) it gives:
 36.917972 seconds (473.90 M allocations: 41.024 GiB, 16.59% gc time, 0.01% compilation time)

The runs are essentially exactly the same, with the final energies equal to each other up to 15 digits.
I bet it is just a matter of a contraction being done in a non-optimal way somewhere (i.e. a non-optimal contraction sequence). I will try to investigate if I have some time, but at least now the the old implementation is available so we can use that in the meantime while we track down where the performance bug is.

Just a simple question then:
if I write the code A = B * C * D ..., or A *= B * C * D, in which order is the contraction performed by ITensor?

You can enable automatic optimization with ITensors.enable_contraction_sequence_optimization(), but we don't have it on by default since it has some overhead (and in general scales exponentially with the number of tensors in the contraction).
Glancing through the code, I do see some parts with non-optimal sequences so hopefully it is a simple fix of reordering some contractions.

For example lines like this:
https://github.com/ITensor/ITensorInfiniteMPS.jl/blob/195bbf9880e8c78e0d43264bb28689c1243494af/src/vumps_nonlocalham.jl#L51
I assume should be:
        temp_Hᶜʰv = temp_Hᶜʰv * ψ.AL[n - j + k] * ∑h[n - j][idx] * ψ′.AL[n - j + k]

That is precisely what I had in mind, yes.


feat: remove deprecated Evented API usage from the public session service

Removes deprecated Evented API usage from the public session service.
handleAuthentication and handleInvalidation methods should be used instead.

Example usage of handleAuthentication https://github.com/mainmatter/ember-simple-auth/blob/master/guides/managing-current-user.md

I'm not sure what I am doing wrong, but handleAuthentication never gets called for me. I used to use this.session.on('authenticationSucceeded', which worked great, but now I have nothing that works.

@RobbieTheWagner
The only thing I could think of is not calling the setup method https://github.com/mainmatter/ember-simple-auth/blob/master/guides/upgrade-to-v4.md#call-sessionsetup-method-in-your-applicationroute

I am calling the setup method and have been for a long time, so it's not that.

Other scenario could be misplaced handleAuthentication method.
I remember it used to exist for an application route at some point but was removed with mixins and the service should be extended instead.
I don't see how else it could break other than the service not existing at all.
You could inspect the classic-test-app and test-app to see how to use session service now. This PR specifically that removes mixins  https://github.com/mainmatter/ember-simple-auth/pull/2480/files
The event logic was also removed from the public service only while the internal one still has them and they are actually being used for triggering the handleAuthentication method.
We didn't change much in terms of internal logic so far, just the public interface.
Technically you could do this.session.session.on('authenticationSucceeded') and it should work in the same way.
It's hard to talk about this without any code, but everything should work the same as long as mixins were correctly replaced with the session methods.

@BobrImperator this is the PR updating things. It has a failing test because handleAuthentication is never called https://github.com/shipshapecode/swach/pull/1732
We do use https://github.com/adopted-ember-addons/ember-cognito and I am not sure if it needs changes to work with latest?

I made a PR for Swach, please see: https://github.com/shipshapecode/swach/pull/1736

Hi @hoIIer
The Evented API had been removed from the service, a direct replacement would be extending the service and implementing a handleAuthentication and handleInvalidation.
e.g. Test app example
import { inject as service } from '@ember/service';

import Session from 'ember-simple-auth/services/session';

export default class SessionService extends Session {

  handleAuthentication() {
    super.handleAuthentication(...arguments);

    // Do you thing here
  }
}

Although if your only requirement is to render a specific thing on authenticated session, then you should be able to use the service's isAuthenticated property.
{{!-- app/templates/application.hbs --}}

  …
  {{#if this.session.isAuthenticated}}
    Logout
  {{else}}
    {{#link-to 'login'}}Login{{/link-to}}
  {{/if}}

  {{outlet}}

See Walkthrough and Other guides in the readme
Lastly

Also the event api still in the docs?

That appears to be a mistake, since the public SessionService doesn't emit anything and only hooks itself up to the internal, private one.


Add right click Duplicate File context menu option to File Explorer

Reopening because this was closed without any resolution: https://github.com/microsoft/vscode/issues/127256
Problem statement
Quite often developers start writing code from existing code. Especially for templated code like test cases or REST apis. It currently takes 4 steps to do this in VSCode (copy, paste, find, rename). In macOS finder you can right-click on any file and select duplicate and the file will be copied/duplicated with the suffix -copy added to the name. It would be great to have this functionality as a context menu item in the explorer of VSCode.
Existing workflow
If you want to make a copy of a file in VSCode today, you must right-click and select copy but you cannot right-click and select paste. No, you have to find an empty area of the explorer window or the folder name that you want the file in to right-click and select paste. Then you need to find the file that just got pasted, and right-click a third time and select rename to rename the file. This is four (4) steps just to duplicate one file. This is an extremely clumsy user experience.
Proposed workflow
A developer can highlight any file in the explorer and right-click and select duplicate. The command pallet prompt area would open up with the name of the file requesting the name of the new file. The developer would name the file and press enter and have a new file with a new name with the contents of the original file. I believe this is a much better user experience.
Workarounds
Ctrl+Drag on Windows. Option+Drag on Mac. But I don't think anyone really knows about these. We use Duplicate File a lot, it should be in the context menu with a hint for the shortcut.
Benefit to developers
Developers can duplicate a file with a new name in a single interaction instead of having to perform four steps (copy, paste, find the file, rename)

I suggest you look for an extension that does this.
https://marketplace.visualstudio.com/search?term=duplicate file&target=VSCode&category=All categories&sortBy=Relevance

Thank you. Since I found the shortcut, I'll be using it.
For reference, XCode has File > Duplicate right under the save button.

Hope you guys will add it somewhere more accessible for new users.


Fix nullptr when mix mode change to only v3.

What problem does this PR solve?
Issue Number: close #4893
Problem Summary:

When we changed MIX_MODE to ONLY_V3, we will change the run_mode at last.
Then we will used PageReaderImplMixed/PageWriterImplMixed to read/write page. But in that ps_v2 will be nullptr.

What is changed and how it works?

make writer and reader not used Mixed, if we do change MIX_MODE to ONLY_V3.

Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/merge


NIFI-12173 - Nifi Registry there's no documentation about OIDC connection

Summary
NIFI-12173
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[ ] Apache NiFi Jira issue created

Pull Request Tracking

[ ] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-12173
[ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-12173

Pull Request Formatting

[ ] Pull Request based on current revision of the main branch
[ ] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[ ] Build completed using mvn clean install -P contrib-check

[ ] JDK 21

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

@exceptionfactory I'm done with it!


Ensure access token is always current.

What's this PR do?
This pull request will check a user's access token on page load, and refresh it if it's expired. This is important because it lets Northstar know accurate activity information (to within 1 hour), which is necessary for tracking our MAU goal.
How should this be reviewed?
We should make sure this doesn't have any unexpected cases where a user will get logged out unexpectedly, and that it doesn't have any performance impact for normal page loads.
Any background context you want to provide?
💹
Relevant tickets
…
Checklist

[ ] Documentation added for new features/changed endpoints.
[ ] Tested on staging.
[ ] Pinged a PM if this is a larger PR that would benefit from some additional testing love
[ ] Post a message in #phoenix if this includes something that causes a rebuild!

Yeah I'm hoping not, since the $user is already loaded in memory and at that point it's just a quick integer comparison on the timestamps but we'll see! Definitely something to monitor in New Relic.


Minor edit to remove some references to statics.

NetflixOSS » eureka » eureka-pull-requests #459 SUCCESS
This pull request looks good

@aroger-r7 fyi some minor edits.


Support exception handling on windows

This was a tough one, but I learned a lot along the way and I'm glad I accomplished it.

Wat?  You are ****ing amazing, Chris!

My hero!
Great work :tada:

@RX14 💯 💯 💯 💯 💯 💯 💯

A lot of this work was @bcardiff and @txe, I stand on the shoulders of giants.

I used a compiler with this patch to compile my local branch with Windows library bindings. It didn't break anything, but the specs don't contain any expectations about exceptions, because that didn't work until now. But I'll work on that.
I also compiled crinja's spec suite. It's one of the largest Crystal projects I know and there are quite a number of exception raised and rescued. And it worked fine :+1: Not all specs passed (which was to be expected), but none fails because of an error in exception handling.

@straight-shoota thats fantastic. I wouldn't be surprised if there's still quite a few edge cases in here though. The key hint is usually an "invalid instruction" segfault (llvm adds a ud2 instruction at the end of catch blocks where it thinks it should be unreachable) or a module validation failure.

Should this also be on 0.26.0?

Pushed an update which merges the old and new exception handling code and improves the comments.

Added a comment @bcardiff


feat: use single decorator for nlp crafters

@JoanFM Let's use the multistage Dockerfile to avoid installing pytest.

can we do in another iteration?


[META] Project overview.

This post is intended to clarify and set the direction of where Scriptchat (SCRIPTCHAT?) is going. This reflects roughly my thoughts at the moment but is still early enough to be open to discussion and interpretation.
Architecture
This is a client-side only single-page app bootstrapped with create-react-app. It's React/Redux-based, which is my go-to framework of choice, in a "If all you have is React/Redux then every problem looks like a nail" sort of way. I am open to reasonable proposals to adopt another framework if a sufficient case can be made.
There is no server-side component. An early experiment of this tried a server-side component as its own chat server, but this seems like overkill and you would never get people to use it, which is why we use Slack transcripts now. Once we're only parsing existing logs, this can all be done client-side, which means (a) hosting is free (b) we don't need to worry about keeping private information safe over the wire or on the back-end side. Also, since the view will eventually be in-browser, this simplies the user interaction dramatically, as we won't require users to download, install, or run any command lines things to check it out. All you need is access to the exports from a Slack channel.
User flow
It basically looks like this:

Add Slack logs.
Get a script.
2b. Possibly tweak it a little.
Export / save it somewhere, if you want.

1. Add Slack logs.
Slack logs can be obtained by any owner of a Slack channel (see issue #2). At minimum, one would need two files: (1) the transcript itself (a JSON file), and (2) the users.json file, which contains data about all the users in that channel. Putting these two together is sufficient to generate a screenplay.
Transcripts are broken up by channel, then by day. We may need to think through how to handle screenplays that span multiple days.
The users file contains information about the name of the user. It doesn't contain other things that screenplays usually need to know like gender or physical description. We may need to think through how to add or infer that information.
The formats are unique enough that Scriptchat should be able to understand what data is being added. There should only need to be one "add a file" button (or drag-drop onto the window to add) -- you wouldn't need separate buttons for each type. Scriptchat may either decide to maintain a collection of different files or just overwrite the previous one when a new one is added.
Providing helpful errors is a good idea at this stage:

When a JSON file being dropped cannot be understood as either a chat log or users file.
When a chat log and users file doesn't seem to match.
Reject any file that cannot be read as text (e.g. binary).

Text files may be Fountain based, in which case we pass it straight through to Fountain for display, since we get it "for free." This makes it easy for an exported script file to be loaded through the same interface later.
2. Get a script.
Once the two Slack files are uploaded, processing happens. This step is entirely done in code. Data in, data out. This pipeline is good for anyone who knows JavaScript but doesn't want to meddle with the browser part of things. It is especially good if you like functional programming and writing tests. This is our "secret sauce," as the quality of this pipeline influences the quality of the screenplay.
Roughly, these things need to happen.

Process users.json:

Each user is an object in an array. This means looking up any user needs to iterate over the array and check the id property; this could get expensive. If we rewrite the data structure so that the id is a key lookup, matching users to the transcript gets easier to do.
Surface the useful information: first name and last name is the most important right now. Slack does not seem to use the same properties for everyone (it depends on what data the user provided) so we need to do some normalization.
Not all information is relevant and it may be dropped.

Convert chat log to Fountain tokens. Fountain.js uses a token structure internally to handle the translation between parsing a Fountain-formatted text file and the HTML view. It is not a public API but because it works, we should steal it. Holding our data state in this token structure allows us to work on it programatically.
Process tokens:

User IDs need to be replaced with real names. Use CAPITAL LETTERS and full names (if we have it) when a character is first introduced in any action. Use just the first name when spoken by another character.
Consecutive messages from the same user should be consolidated.
Emojis that are present and look like emotion markers, rather than grammatical entities, should be extracted and converted to parentheticals or actions.
People entering or exiting the channel are actions.
We can really get super creative here; this is just the obvious ones, but imagine what else you can do here.

Output tokens. When processing is done, the tokens are shuffled off to a place in the Redux store, where the React view takes over.
Convert tokens to HTML markup. This is entirely ripped from Fountain.js and really needs very little work from us.

Also, title page
There is a special part of the Fountain syntax reserved for the title page. The view is essentially taken care of, but to populate it, there is a set of key-value pairs that make it up. It's possible that in the parsing stage we can infer some of these, but it's also easier on the user to expose the UI for it so that it can be customized.
Here is a list of all the supported title page keys: https://github.com/mattdaly/Fountain.js/blob/master/fountain.js#L225-L234
We'll incorporate that into the UI. Speaking of:
2b. Possibly tweak it a little.
In addition to editing title page properties, it's possible that the script itself might need some tweaking. Earlier, I mentioned that it would not be possible to obtain gender pronouns from Slack. We may need to provide UI after the fact to add those to users, plus other identifying information or characteristics that would enliven the script.
This step is extremely open to interpretation. Also, we may not know exactly what tweakable knobs we need to present until doing a number of test runs to figure out what really needs some human intervention.
3. Export / save it somewhere.
Once you have a script, it should go somewhere. Exporting as Fountain format is a must, since it can be read by other screenplay writing tools like Final Draft. Exporting as PDF is just as important, because that's easier to send around. Generating either of these can be done by using the token structure we have and writing that to a file.
And that's Scriptchat! In a future phase, we put on an actual production!

❤ Fountain


Update auto-generated documentation

Repository: openmicroscopy/ome-documentation
Already up-to-date.
Generated by build OMERO-DEV-latest-docs-autogen#609.

--no-rebase

Merging before tagging

👍


fix issue #290

PR Details

Description

Related Issue

Motivation and Context

How Has This Been Tested

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Codecov Report

Merging #291 into master will decrease coverage by 0.05%.
The diff coverage is 33.33%.

@@            Coverage Diff             @@
##           master     #291      +/-   ##
==========================================
- Coverage   98.99%   98.94%   -0.06%
==========================================
  Files          18       18
  Lines        3895     3896       +1
==========================================
- Hits         3856     3855       -1
- Misses         23       24       +1
- Partials       16       17       +1

Impacted Files
Coverage Δ

cell.go
98.19% <33.33%> (-0.6%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2aca340...ef334ee. Read the comment docs.


The new shared library causes conflicts if more than 1 variant of llama-cpp-python is imported

The ctypes.CDLL call under https://github.com/abetlen/llama-cpp-python/blob/7e20e346bd49cc8f0031eb053fe879a38c777b6f/llama_cpp/llama_cpp.py#L75 loads symbols in the global scope. In my project, it is possible to load 3 different versions of llama-cpp-python:

CUDA
CUDA + tensorcores (without  -DGGML_CUDA_FORCE_MMQ=ON)
CPU

Due to the shared library, when one of the libraries is already imported and the user switches to another one, undefined behavior happens, such as the CPU version having BLAS=1 in its logs.
Can this be prevented? I think that it may be impossible to work around this issue on my side. I have tried importlib.reload and it didn't work.

@oobabooga and this only became an issue recently. Yes I had to make some changes to how the shared library is loaded because of the recent refactor PRs.

@oobabooga can you try out this PR https://github.com/abetlen/llama-cpp-python/pull/1576

Same issue after the changes. I'm on Linux, so the only line that applies is
https://github.com/abetlen/llama-cpp-python/pull/1576/files#diff-9184e090a770a03ec97535fbef520d03252b635dafbed7fa99e59a5cca569fbcR26
I have tried loading a model with my llama_cpp_cuda (CUDA version), then unloading and loading it with llama_cpp (CPU version), and I still got BLAS = 1 in the logs after adding that line.

@oobabooga that's definitely bizarre, the changes I made recently were just to account for the recent llama.cpp refactor PR that made ggml a shared library as opposed to a static link. To account for this I had to update the rpath of the libllama library to check the lib directory for libggml.

I don't understand what happened either. After compiling 0.2.81 and updating from 0.2.79, I started getting undefined symbol errors such as
undefined symbol: ggml_backend_cuda_log_set_callback

after importing all libraries at the same time, like
import llama_cpp
import llama_cpp_cuda
import llama_cpp_cuda_tensorcores

I updated my workflow to replace absolute imports in llama-cpp-python like
import llama_cpp
from llama.cpp.llama_cpp import ...

with relative ones, and ended up with a robust code for the replacement:
https://github.com/oobabooga/llama-cpp-python-cuBLAS-wheels/blob/891ea4e8c8115e2a69b6dece49081dd78e966b5f/.github/workflows/build-wheels-oobabooga.yml#L187
But it still didn't work. I also tried monkey patching ctypes.CDLL to always use mode=RTLD_LOCAL, and that also didn't work. There is some leak between the different libraries, and reimporting the library before using it with importlib.reload doesn't work either. My guess was that symbols were leaking due to the shared library import, but maybe it's something else.

Also, about
undefined symbol: ggml_backend_cuda_log_set_callback

the error didn't appear if I loaded only one of the libraries at a time, and they worked fine when loaded alone. The issue happened when more than 1 library was imported in sequence.

Any workaround in https://github.com/oobabooga/text-generation-webui/issues/6202 to solve this ? my mac stopped worked for 4 days already... I need to use GGUF format..

@poopro One could try to downgrade back to a version without this problem with or without the corresponding Python packages. However, I don't know the latest version which didn't have this problem.

This is a convenience issue, you can use the library just fine, you just can't switch between the "cpu", the "tensorcores", and the default setting (without those two) without restarting the server. But each of those works fine by itself.

@oobabooga Sorry if this is a seperate issue. I actually mean that a model cannot be loaded on the GPU, because of this bug. In fact I cannot use any Meta-Llama-3 model (out of the 4 I have) on my GPU now all of a sudden. The error doesn't happen when only using the CPU. See https://github.com/oobabooga/text-generation-webui/issues/6202


Support for custom generators / targets

I am interested in helping provide a feature!
Yes
Which generators are impacted?

[X] All
[ ] Angular
[ ] HTML
[ ] Qwik
[ ] React
[ ] React-Native
[ ] Solid
[ ] Stencil
[ ] Svelte
[ ] Vue
[ ] Web components

What problem does this feature solve?
Some projects may need to customize, rewrite or write generators from scratch in order to achieve some specific business requirements that are not compatible with current coding standards.
What does the proposed API look like?
From an initial high level investigation it should be possibile to load custom targets directly from the config file, in the cli package it should be enough to merge the current targets with the ones from the options. This should also enable generator replacements.
Additional Information
No response

I'm open to adding this! How to do so:

Add a generators dictionary prop to the MitosisConfig:
https://github.com/BuilderIO/mitosis/blob/e862056eca675887bd975bf5696c61666d0f3532/packages/core/src/types/config.ts#L19

Update this call to take into account options.generators, by checking for options.generators[target] before relying on the default generators:
https://github.com/BuilderIO/mitosis/blob/e862056eca675887bd975bf5696c61666d0f3532/packages/cli/src/build/build.ts#L278

I will do it


fix: add typeVersions to package.json

fixes the following type error

Can you show your tsconfig please? this shouldn't happen if you're using ESM + Node16 moduleResolution

Can you show your tsconfig please? this shouldn't happen if you're using ESM + Node16 moduleResolution

oops completely forgot about that! thank you!


Update test file for Lotusrc class

Use existing @lotusrc instead Lotus::Lotusrc.new(@root);
Add missing test for checking returned options keys;

@davydovanton can you please update the description what the refactoring is?

@joneslee85 yes, of course

@davydovanton because you are refactoring this test, can you extract to a before block below https://github.com/lotus/lotus/pull/446/files#diff-b83ba0c0465de991a609fb85be6dd33dR4 the common things except the Dir.chdir 'path' line? to avoid duplicated code. what do you think?

@AlfonsoUceda done. I replaced all before block but for each case I created different path to lotusrc file. What do you think? This is better for you?

@davydovanton could you do the same for after block? then :+1: to me, thank you so much ;)

oh, sorry I did not notice after block :disappointed:
Done too

@AlfonsoUceda could you stop current travis build? (https://travis-ci.org/lotus/lotus/builds/100646008) this will save our time :smiley:

haha don't worry ;) stopped

nice, thank you very much @AlfonsoUceda! :tada:

@davydovanton merging, because this is only a refactor in a test. thanks!!
cc @lotus/core-team

Thanks for merge!


Fully Deployed

Website is functional and meets AC

missing a clear button and localStorage isn't quite persistent yet... but website meets AC


Multicluster two per network

Per @nmittler on https://github.com/istio/istio/pull/23732

In the future, we might consider extending it to 4 clusters. In this case the first 2 would be same network with master/remote, and the second 2 would be on a different network with master/remote. That way we also test dual masters on separate networks.

Start multicluster tests faster by creating Kind clusters in parallel
Run a fourth cluster so the topology has clusters 1 &2 on the first network, and clusters 2 & 3 on the other.
Make multicluster/remote use one cluster from each network as a master cluster

[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[X] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

integ-galley-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-galley-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

integ-galley-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-galley-k8s-tests_istio

integ-multicluster-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

integ-galley-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-galley-k8s-tests_istio

integ-multicluster-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-multicluster-k8s-tests_istio

integ-mixer-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-mixer-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

integ-galley-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-galley-k8s-tests_istio

integ-multicluster-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-multicluster-k8s-tests_istio

integ-mixer-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-mixer-k8s-tests_istio

release-test_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test release-test_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

integ-galley-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-galley-k8s-tests_istio

integ-multicluster-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-multicluster-k8s-tests_istio

integ-mixer-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-mixer-k8s-tests_istio

release-test_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test release-test_istio

integ-pilot-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

integ-galley-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-galley-k8s-tests_istio

integ-multicluster-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-multicluster-k8s-tests_istio

integ-mixer-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-mixer-k8s-tests_istio

release-test_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test release-test_istio

integ-pilot-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-pilot-k8s-tests_istio

integ-distroless-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-distroless-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

integ-galley-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-galley-k8s-tests_istio

integ-multicluster-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-multicluster-k8s-tests_istio

integ-mixer-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-mixer-k8s-tests_istio

release-test_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test release-test_istio

integ-pilot-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-pilot-k8s-tests_istio

integ-distroless-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-distroless-k8s-tests_istio

integ-security-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-security-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-telemetry-k8s-tests_istio

integ-local-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-local-tests_istio

integ-operator-controller-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-operator-controller-tests_istio

integ-galley-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-galley-k8s-tests_istio

integ-mixer-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-mixer-k8s-tests_istio

release-test_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test release-test_istio

integ-pilot-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-pilot-k8s-tests_istio

integ-distroless-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-distroless-k8s-tests_istio

integ-security-k8s-tests_istio
520ff54a80700b93745cf4086303fdfbd83c988a
link
/test integ-security-k8s-tests_istio

integ-multicluster-k8s-tests_istio
26f8049e25f50703ca4bf713511764e7ec5abfa9
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
26f8049e25f50703ca4bf713511764e7ec5abfa9
link
/test integ-multicluster-k8s-tests_istio

lint_istio
8f3665d07864f7b478f28de854abe5429cd5b37a
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8f3665d07864f7b478f28de854abe5429cd5b37a
link
/test lint_istio

integ-multicluster-k8s-tests_istio
8f3665d07864f7b478f28de854abe5429cd5b37a
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

/test

@stevenctl: The /test command needs one or more targets.
The following commands are available to trigger jobs:

/test unit-tests_istio
/test release-test_istio
/test integ-galley-k8s-tests_istio
/test integ-mixer-k8s-tests_istio
/test integ-pilot-k8s-tests_istio
/test integ-security-k8s-tests_istio
/test integ-telemetry-k8s-tests_istio
/test integ-multicluster-k8s-tests_istio
/test integ-distroless-k8s-tests_istio
/test integ-operator-controller-tests_istio
/test integ-local-tests_istio
/test lint_istio
/test gencheck_istio

Use /test all to run all jobs.

In response to this:

/test

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/test all

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8f3665d07864f7b478f28de854abe5429cd5b37a
link
/test lint_istio

integ-multicluster-k8s-tests_istio
8c86d0c9cdd41a06636d55e7901ac214870bd10d
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-multicluster-k8s-tests_istio

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
a58fd01aaa329646460a825ca6b5db35c571c0aa
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-multicluster-k8s-tests_istio

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
14ec5d229d1297ccea91c5d4c428a6400190ade2
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-multicluster-k8s-tests_istio

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
1e7859c6b8cac8f21ee2adced5391afb97da6243
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-multicluster-k8s-tests_istio

Tried to make the kind clusters get spun up in parallel but it had some issues.

/test

@stevenctl: The /test command needs one or more targets.
The following commands are available to trigger jobs:

/test unit-tests_istio
/test release-test_istio
/test integ-galley-k8s-tests_istio
/test integ-mixer-k8s-tests_istio
/test integ-pilot-k8s-tests_istio
/test integ-security-k8s-tests_istio
/test integ-telemetry-k8s-tests_istio
/test integ-multicluster-k8s-tests_istio
/test integ-distroless-k8s-tests_istio
/test integ-operator-controller-tests_istio
/test integ-local-tests_istio
/test lint_istio
/test gencheck_istio

Use /test all to run all jobs.

In response to this:

/test

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/test all

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-mixer-k8s-tests_istio
cd50fd2e501c0eced53c3865a40f0072e2e6b0ee
link
/test integ-mixer-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-mixer-k8s-tests_istio
cd50fd2e501c0eced53c3865a40f0072e2e6b0ee
link
/test integ-mixer-k8s-tests_istio

integ-multicluster-k8s-tests_istio
cd50fd2e501c0eced53c3865a40f0072e2e6b0ee
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-mixer-k8s-tests_istio
cd50fd2e501c0eced53c3865a40f0072e2e6b0ee
link
/test integ-mixer-k8s-tests_istio

integ-multicluster-k8s-tests_istio
410bb3ed15d8e1161bd92f970cb261a54dc701ed
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-mixer-k8s-tests_istio
cd50fd2e501c0eced53c3865a40f0072e2e6b0ee
link
/test integ-mixer-k8s-tests_istio

integ-multicluster-k8s-tests_istio
d483bebe409485a2b41d5d4bfd621c6641a58f4f
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
3e39667e48725cb0063645409ea9a43bd7608cf7
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
3e39667e48725cb0063645409ea9a43bd7608cf7
link
/test lint_istio

integ-multicluster-k8s-tests_istio
3e39667e48725cb0063645409ea9a43bd7608cf7
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
3e39667e48725cb0063645409ea9a43bd7608cf7
link
/test integ-multicluster-k8s-tests_istio

lint_istio
bfa41298c32fec0ab4dfa7e7b273e4209b37f6a6
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
bfa41298c32fec0ab4dfa7e7b273e4209b37f6a6
link
/test lint_istio

integ-multicluster-k8s-tests_istio
bfa41298c32fec0ab4dfa7e7b273e4209b37f6a6
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
bfa41298c32fec0ab4dfa7e7b273e4209b37f6a6
link
/test integ-multicluster-k8s-tests_istio

lint_istio
61c3e71679ecd17bb75c61cec3168917089ac2c8
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
61c3e71679ecd17bb75c61cec3168917089ac2c8
link
/test lint_istio

integ-multicluster-k8s-tests_istio
61c3e71679ecd17bb75c61cec3168917089ac2c8
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
cf1b1d3dee0ff438e4f5e5b4e674821eb204d344
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
cf1b1d3dee0ff438e4f5e5b4e674821eb204d344
link
/test integ-pilot-k8s-tests_istio

lint_istio
6c6321f803bb5d9ca3000bb4b3b7409f4e0d84e6
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
cf1b1d3dee0ff438e4f5e5b4e674821eb204d344
link
/test integ-pilot-k8s-tests_istio

lint_istio
6c6321f803bb5d9ca3000bb4b3b7409f4e0d84e6
link
/test lint_istio

integ-distroless-k8s-tests_istio
6c6321f803bb5d9ca3000bb4b3b7409f4e0d84e6
link
/test integ-distroless-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Istio setup turns out to be quick
=== DONE: Setup: 'multicluster_multimaster' (41.499709905s) ===

=== DONE: Setup: 'multicluster_remote' (49.870446907s) ===

Going to try to find efficiency in the tests themselves.

That's great!

Compared with a recent run of multicluster tests, the prow setup was close to the same within 30s or so:

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-mixer-k8s-tests_istio
1cdd77ed9b826bee4bde31bcfa7bd84366621205
link
/test integ-mixer-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-mixer-k8s-tests_istio

/test integ-multicluster-k8s-tests_istio

/test integ-pilot-multicluster-tests

@stevenctl: The specified target(s) for /test were not found.
The following commands are available to trigger jobs:

/test unit-tests_istio
/test release-test_istio
/test integ-galley-k8s-tests_istio
/test integ-mixer-k8s-tests_istio
/test integ-pilot-k8s-tests_istio
/test integ-security-k8s-tests_istio
/test integ-telemetry-k8s-tests_istio
/test integ-multicluster-k8s-tests_istio
/test integ-distroless-k8s-tests_istio
/test integ-ipv6-k8s-tests_istio
/test integ-operator-controller-tests_istio
/test lint_istio
/test gencheck_istio

Use /test all to run all jobs.

In response to this:

/test integ-pilot-multicluster-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/test integ-pilot-multicluster-tests_istio_postsubmit

@stevenctl: The specified target(s) for /test were not found.
The following commands are available to trigger jobs:

/test unit-tests_istio
/test release-test_istio
/test integ-galley-k8s-tests_istio
/test integ-mixer-k8s-tests_istio
/test integ-pilot-k8s-tests_istio
/test integ-security-k8s-tests_istio
/test integ-telemetry-k8s-tests_istio
/test integ-multicluster-k8s-tests_istio
/test integ-distroless-k8s-tests_istio
/test integ-ipv6-k8s-tests_istio
/test integ-operator-controller-tests_istio
/test lint_istio
/test gencheck_istio

Use /test all to run all jobs.

In response to this:

/test integ-pilot-multicluster-tests_istio_postsubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Load images actually takes up to 8 mins, even with 3 clusters:
https://prow.istio.io/view/gcs/istio-prow/pr-logs/pull/istio_istio/24549/integ-pilot-multicluster-tests_istio/14

We can consider using https://kind.sigs.k8s.io/docs/user/local-registry/. I
tried it out a while back, might be able to dig up the PR
On Wed, Jun 24, 2020 at 11:40 AM Steven Landow<EMAIL_ADDRESS>wrote:

Load images actually takes up to 8 mins, even with 3 clusters:
https://prow.istio.io/view/gcs/istio-prow/pr-logs/pull/istio_istio/24549/integ-pilot-multicluster-tests_istio/14
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/23969#issuecomment-648996822, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEYGXNFTPFWTL3UFXCZ6TLRYJCBDANCNFSM4NFJ4Z3A
.

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-ipv6-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-ipv6-k8s-tests_istio

integ-security-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-security-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-ipv6-k8s-tests_istio

integ-security-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-distroless-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-ipv6-k8s-tests_istio

integ-security-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-distroless-k8s-tests_istio

integ-operator-controller-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-operator-controller-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-ipv6-k8s-tests_istio

integ-security-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-distroless-k8s-tests_istio

integ-operator-controller-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-operator-controller-tests_istio

integ-pilot-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-ipv6-k8s-tests_istio

integ-security-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-distroless-k8s-tests_istio

integ-operator-controller-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-operator-controller-tests_istio

integ-pilot-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-pilot-k8s-tests_istio

integ-telemetry-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-ipv6-k8s-tests_istio

integ-security-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-distroless-k8s-tests_istio

integ-operator-controller-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-operator-controller-tests_istio

integ-pilot-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-pilot-k8s-tests_istio

integ-telemetry-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-telemetry-k8s-tests_istio

unit-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

lint_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test lint_istio

integ-mixer-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-mixer-k8s-tests_istio

integ-galley-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-galley-k8s-tests_istio

integ-ipv6-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-ipv6-k8s-tests_istio

integ-security-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-security-k8s-tests_istio

integ-distroless-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-distroless-k8s-tests_istio

integ-operator-controller-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-operator-controller-tests_istio

integ-pilot-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-pilot-k8s-tests_istio

integ-telemetry-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-telemetry-k8s-tests_istio

unit-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test unit-tests_istio

integ-multicluster-k8s-tests_istio
8cc2168011fe2cad2e964399bf30bd6ddfa8a16d
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
e457f5e6c579f78845088607ebd3884040d0d20d
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
53a8eaadcb123fa9b5f597f0a8a48132025c6469
link
/test analyze-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stevenctl: PR needs rebase.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@stevenctl: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

analyze-tests_istio
53a8eaadcb123fa9b5f597f0a8a48132025c6469
link
/test analyze-tests_istio

release-notes_istio
2103cac62b22d565d199661912cac0806f775f58
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn This finishes in under 20 minutes, compared with the current state of things which takes ~24 mins. We might be able to squeeze a little bit more if we make centralistiod only utilize 3 clusters (1 primary, one same-network, one different-network) but that will most-likely be marginal.
We can also do suite-level echos across the reachability/cluster-local/telemetry tests for extra gains.