Module dependency version differences

(This is continuation / expansion of #112 which was closed without any explanation.)
Looking at the latest main branch code...
For some components, there are different dependency versions listed in requirements.txt files:
$ find -name requirements.txt | xargs grep -h == | sort -u
frontend==0.0.3
haystack-ai==2.2.4
huggingface_hub==0.24.0
langchain #==0.1.12
langchain==0.1.16
langchain==0.2.12
langchain==0.2.9
langchain-community==0.2.1
langchain_community==0.2.5
langchain_community==0.2.7
langchain-core==0.2.21
langchainhub==0.1.20
llama-index-embeddings-huggingface==0.2.0
lm-eval==0.4.2
motor==3.4.0
openai==1.35.13
pgvector==0.2.5
prometheus-fastapi-instrumentator==7.0.0
pydantic #==1.10.13
pydantic==2.7.2
pydantic==2.7.3
pydantic==2.8.2
pymilvus==2.4.3
pymupdf==1.24.5
python-bidi==0.4.2
python-docx==0.8.11
sentence-transformers==3.0.1
torch==1.13.1 --index-url https://download.pytorch.org/whl/cpu
torchaudio==0.13.1 --index-url https://download.pytorch.org/whl/cpu
torchvision==0.14.1 --index-url https://download.pytorch.org/whl/cpu
unstructured[all-docs]==0.11.5

Both for langchain and langchain_community:
$ find -name requirements.txt | xargs grep langchain.*==
./comps/agent/langchain/requirements.txt:langchain==0.2.9
./comps/agent/langchain/requirements.txt:langchain_community==0.2.7
./comps/agent/langchain/requirements.txt:langchainhub==0.1.20
./comps/knowledgegraphs/requirements.txt:langchain_community==0.2.5
./comps/ragas/tgi/requirements.txt:langchain==0.1.16
./comps/dataprep/redis/langchain/requirements.txt:langchain==0.2.12
./comps/llms/faq-generation/tgi/requirements.txt:langchain==0.1.16
./comps/llms/text-generation/vllm/requirements.txt:langchain==0.1.16
./comps/llms/text-generation/ollama/requirements.txt:langchain==0.1.16
./comps/llms/text-generation/ray_serve/requirements.txt:langchain==0.1.16
./comps/llms/text-generation/vllm-ray/requirements.txt:langchain==0.1.16
./comps/llms/text-generation/vllm-xft/requirements.txt:langchain==0.1.16
./comps/llms/summarization/tgi/requirements.txt:langchain #==0.1.12
./comps/llms/requirements.txt:langchain==0.1.16
./comps/lvms/video-llama/server/requirements.txt:langchain==0.2.9
./comps/lvms/video-llama/server/requirements.txt:langchain-community==0.2.1
./comps/lvms/video-llama/server/requirements.txt:langchain-core==0.2.21

And pydantic:
$ find -name requirements.txt | xargs grep pydantic.*==
./comps/retrievers/langchain/milvus/requirements.txt:pydantic==2.7.3
./comps/agent/langchain/requirements.txt:pydantic #==1.10.13
./comps/asr/requirements.txt:pydantic==2.7.2
./comps/finetuning/requirements.txt:pydantic==2.8.2
./comps/llms/utils/lm-eval/requirements.txt:pydantic==2.7.2
./comps/lvms/requirements.txt:pydantic==2.7.2

Is this intentional, or just a bug?

This is somewhat related to #265.

@eero-t Thanks for reporting this. Currently the new examples are enabled gradually and keep added to new releases, each of the examples might be based on different base library versions during developing and evolving due to different time frame. The Docker images are separate for all use cases. And we try to make sure each of the examples works.
@kevinintel Do we have plans to align all the versions of the common dependencies for all examples, such as langchain, langchain-community and pydantic?

@kevinintel Could you please comment on this issue? Thanks.
The OP has some concerns that each example or each microservices components inside the example uses different versions of base library such as langchain, further putting obstacles into creating a common based docker images for those components to reduce the image size.

@eero-t Thanks for reporting this. Currently the new examples are enabled gradually and keep added to new releases, each of the examples might be based on different base library versions during developing and evolving due to different time frame. The Docker images are separate for all use cases. And we try to make sure each of the examples works.
@kevinintel Do we have plans to align all the versions of the common dependencies for all examples, such as langchain, langchain-community and pydantic?

if this is a community project, version would be different

if this is a community project, version would be different

@kevinintel What do you mean?
Current Dockerfiles are awful.  They should look something like this:
FROM opea/components-base
COPY some.py /home/user/
ENTRYPOINT ["python", "some.py"]

Which results in KB sized container (on top of the shared base image), but instead they are monsters each taking nearly GB of network bandwidth, disk space etc.
Making sure that same versions of dependencies work, is one step in fixing that i.e. getting to common base image(s).

@feng-intel Why did you close this?


add fixture 'cameo/gobo-scanner-80'

add fixture 'cameo/gobo-scanner-80'
update register.json

Thank you @Fxedel!

Export files validity
(Output of test script tests/github/exports-valid.js.)
Test the exported files of selected fixtures against the plugins' export tests.
You can run a plugin's export tests by executing:
$ node cli/run-export-test.js -p <plugin name> <fixtures>
Fixture cameo/gobo-scanner-80 changed in this PR

d-light

attributes-correctness

:white_check_mark: cameo/gobo-scanner-80-12ch.xml
:white_check_mark: cameo/gobo-scanner-80-9ch.xml

ecue
ofl
qlcplus

xsd-schema-conformity

:white_check_mark: cameo-Gobo-Scanner-80.qxf


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from  to before v5.14.15",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=f5966ba53013149bcf94e1536644a958dd00a026"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "f5966ba53013149bcf94e1536644a958dd00a026",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "isdn: mISDN: Fix sleeping function called from invalid context\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v5.14.15 by commit f5966ba53013149bcf94e1536644a958dd00a026. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1002117


Uploader.Upload - MediaFireApiError: 128: Required parameters missing

I'm trying to figure out the issue with my code.
I have tried to base my code off of the examples in the uploader(session) and mediafire.MediaFireUploader on the GitHub page.
Code, running python 2.7.11rc1 :
`import os, Tkinter, tkFileDialog, logging, pprint
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
from mediafire import (MediaFireApi, MediaFireUploader)
#Log in
api = MediaFireApi()
session = api.user_get_session_token(app_id='42511',
email='email',
password='password')
api.session = session
uploader = MediaFireUploader(api)
#Get the location of folder with documents to upload
root = Tkinter.Tk()
uploadDirectory = tkFileDialog.askdirectory(parent=root, initialdir="/", title='Please select a directory')
dirs = os.listdir(uploadDirectory)
#Get FolderKey from user for folder to upload into
folderKey = raw_input("What is the folder key? ")
folderKey = "mf:" + folderKey
#Set iteration to see how many files have been evaluated
i=0
#Loop specified directory files for upload
for File in dirs:
flPath = uploadDirectory + "/" + File #Set File Full Path
fd = open(flPath,'rb') #Set file location as object
result = uploader.upload(fd, name=File, folder_key=folderKey, action_on_duplicate="skip")
pprint(api.file_get_info(result.quickkey))
print(i)
i+=1        `
Console Results:
DEBUG:mediafire.uploader:Calculating checksum
DEBUG:mediafire.api:uri=/api/1.3/upload/check.php query=filename=file.pdf&folder_key=mf%3A97ol3i4z62bzu&hash=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855&response_format=json&resumable=no&session_token=13dfda90c6a4d0f59776be4db81120f467c13912bb831a31e7223916178bb846f352d22289f974beaa912ded2d1bada04f4202facdf2c745e3f68222cdfd0bd9d898690c346e7938&size=0&signature=01d87ccaa5494f2cf39bce303d5095c2
INFO:requests.packages.urllib3.connectionpool:Resetting dropped connection: www.mediafire.com
DEBUG:requests.packages.urllib3.connectionpool:"POST /api/1.3/upload/check.php HTTP/1.1" 400 None
DEBUG:mediafire.api:response: {"response":{"action":"upload/check","message":"Required parameters for this request are missing","error":128,"result":"Error","new_key":"yes","current_api_version":"1.5"}}
Traceback (most recent call last):
File "C:\Users\Adam\My Documents\LiClipse Workspace\MediaFireLinks\MediaFireUpload.py", line 32, in
result = uploader.upload(fd, name=File, folder_key=folderKey, action_on_duplicate="skip")
File "C:\Python27\lib\site-packages\mediafire\uploader.py", line 262, in upload
check_result = self._upload_check(upload_info, resumable)
File "C:\Python27\lib\site-packages\mediafire\uploader.py", line 387, in _upload_check
resumable=resumable
File "C:\Python27\lib\site-packages\mediafire\api.py", line 568, in upload_check
'resumable': resumable
File "C:\Python27\lib\site-packages\mediafire\api.py", line 197, in request
return self._process_response(response)
File "C:\Python27\lib\site-packages\mediafire\api.py", line 236, in _process_response
response_node['error'])
mediafire.api.MediaFireApiError: 128: Required parameters for this request are missing

Right away I see you are using mf: prefix when setting the folder, you want to use that prefix only when interacting with the mediafire.client module. Try using client.upload_file or not supplying the mf: prefix to upload() method.

Thank you Roman for noticing that.
I have run the script again and came up with the same result for the MediaFireApiError: 128.  The second DEBUG entry does list the correct folder_key this time however.
I had initially used the client.upload_file and decided to attempt the more specific uploader.upload so I could specify the skip duplicate attribute.
Per the solved issue #18 it looks like the client.upload-file() can accept directories, would I place the directory path in lieu of the file path.
If I were to use the client.upload_file(Path, mf:folder_key), it looks like the action_on_duplicate would be set to 'replace' in client.py.  Am I following that correctly?  If I were to change 'replace' to 'skip' in client.py then I should be able to use client.upload_file to achieve my goal?
Thank you for your time,
Adam

The client will not upload directory contents, #18 is not yet resolved.
I was also able to upload the directory using your script w/o an API error, but I see that your script sent a file size of 0. 0-sized files are a special case in MediaFire API (and that is not handled in the python library) and need to be created via file/create API call.

Success!!!
One of my files had a size of 0 which was causing the issue.
Thank you very much Roman!


Clarification needed: Token indices sequence length is longer ... during inference

Versions
compel==1.1.0
diffusers==0.15.0.dev0

Reproduction code sample:
from diffusers import StableDiffusionPipeline
from compel import Compel

pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder)

prompt = 25 * "a cat playing with a ball in the forest"
negative_prompt = 25 * "ugly, blurry, out of focus "

prompt_embeds = compel.build_conditioning_tensor(prompt)
negative_prompt_embeds = compel.build_conditioning_tensor(negative_prompt)

prompt_embeds, negative_prompt_embeds = compel.pad_conditioning_tensors_to_same_length(
    conditionings=[prompt_embeds, negative_prompt_embeds]
)

images = pipeline(prompt_embeds=prompt_embeds, negative_prompt_embeds=negative_prompt_embeds)

During this, I encounter the following message:
Token indices sequence length is longer than the specified maximum sequence length for this model (227 > 77). Running this sequence through the model will result in indexing errors

So I have those questions:

Do I understand it right, that I can just ignore this?
How can I stop this message from being printed?
Where does it came from?

Thanks in advance

Interesting, i haven't seen that before. Is it happening on one of the calls to compel.build_conditioning_tensor() or does it only appear at pipeline()? In any case because Compel was not initialized with truncate_long_prompts=False, the token sequence ought to be being truncated to the correct length before it gets passed to pipeline (which means the call to pad_conditioning_tensors_to_same_length() is unnecessary).

I'm sorry, I just forgot to paste the truncate_long_prompts=False, into the initialization of compel.
Dependencies:
diffusers==0.14.0
compel==1.1.0

Code, can be run in the colab with GPU:
from diffusers import StableDiffusionPipeline
from compel import Compel
import torch

pipeline = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5",
                                                   safety_checker=None,
                                                   requires_safety_checker=False,
                                                   feature_extractor=None,
                                                   torch_dtype=torch.float16).to('cuda')

compel = Compel(tokenizer=pipeline.tokenizer, text_encoder=pipeline.text_encoder,
                truncate_long_prompts=False)

prompt = 25 * "a cat playing with a ball in the forest"
negative_prompt = 25 * "ugly, blurry, out of focus "

prompt_embeds = compel.build_conditioning_tensor(prompt)
negative_prompt_embeds = compel.build_conditioning_tensor(negative_prompt)

prompt_embeds, negative_prompt_embeds = compel.pad_conditioning_tensors_to_same_length(
    conditionings=[prompt_embeds, negative_prompt_embeds]
)

images = pipeline(prompt_embeds=prompt_embeds, negative_prompt_embeds=negative_prompt_embeds)

Similarly, I have Token indices sequence length is longer than the specified maximum sequence length for this model (227 > 77). Running this sequence through the model will result in indexing errors in the output. I don't understand, because it seems like if I use the truncate_long_prompts=False, I shouldn't see messages like this.

Is the error happening on one of the calls to compel.build_conditioning_tensor() or does it only appear at pipeline()? if it's only happening at pipeline() then this is entirely expected - the long prompt is a hack and it shouldn't work (i don't use it fwiw, it's provided based on user demand)

This happens on compel.build_conditioning_tensor()

i spent some time looking into this, it is in fact an expected warning. token sequences longer than 77 tokens are not a supported usage of the stable diffusion model (i assume you're using SD) - that it works at all is a quirk.

@BEpresent you can pass an argument, i think it's truncate=False, to Compel __init__ and it will turn of truncation

thanks, just for clarification, would the .and() syntax work also with the conditioning tensors and they would be passed in in chunks ?
So instead of this
        conditioning = self.compel.build_conditioning_tensor(long_prompt)
        negative_conditioning = self.compel.build_conditioning_tensor(long_negative_prompt)
        conditioning, negative_conditioning = self.compel.pad_conditioning_tensors_to_same_length([conditioning, negative_conditioning])

Something like this?
        conditioning = self.compel.build_conditioning_tensor(("a b c", "d, e f").and())
        negative_conditioning = self.compel.build_conditioning_tensor(("a b c", "d, e f").and())
        conditioning, negative_conditioning = self.compel.pad_conditioning_tensors_to_same_length([conditioning, negative_conditioning])

close you need to put the '.and()' inside the prompt string:  self.compel.build_conditioning_tensor('("a b c", "d, e f").and()')
compel has methods to count tokens, you can call compel.describe_tokenization() or compel.get_tokens() and check the lenght of the returned arrays. note though that this does not account for syntax - if you call compel.get_tokens() on a prompt like a cat++ playing with a ball the ++ will be included as two extra tokens, whereas when you run compel "for real" they will be applied and removed from the token sequence


My wallet is saying my ZCL is unverified and my BTCP has not arrived

HI Guys
I have had this message in my wallet for the last day or so.

I am also waiting for BTCP Fork to happen and can only assume that the ZCL being unverified is holding this up.
Can you help me figure out why this is and how to fix please?

Mine same way

Have you heard any thing about it? Or if it will still honor btcp

Please email me with any info... thanks   lbereki@ yahoo.com

Ok. I've managed to verify my coins. You'll need to leave your electrum wallet open until the block catches up. Just leave it open and eventually it'll do everything itself. Took about 6 hours. I'm still not sure on if we have to claim or if the fork happens on its own. The fork started 2 hours ago... touch wood there's no issues there


"local and remote addresses" test fails on macOS 11

This has becoming a recurring issue. I first observed it in https://github.com/armanbilge/epollcat/pull/85#issuecomment-1309538957.
So far I've only seen this test failure on macOS, and possibly only macOS 11, but I am less certain of that.
The issue seems to be that getsockname on a client socket sometimes returns 0:0:0:0:0:0:7F00:1. (Is this a valid address for localhost?)
I believe it is should be returning <IP_ADDRESS> aka 0:0:0:0:0:FFFF:7F00:1.
I've pushed a commit that adds debug logging for the output of getsockname in https://github.com/armanbilge/epollcat/commit/8afb7bec5399853600d863a1053c626afde9a024. When I run it, I frequently (but not always) get a test failure like this:
sbt:root> testsNative/testOnly epollcat.TcpSuite
[info] Starting process '/Users/armanbilge/code/epollcat/tests/native/target/scala-2.13/tests-test-out' on port '63434'.
List(28, 30, -9, -52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, -1, 127, 0, 0, 1, 0, 0, 0, 0)
List(28, 30, -9, -52, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -1, -1, 127, 0, 0, 1, 0, 0, 0, 0)
List(28, 30, -9, -51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 127, 0, 0, 1, 0, 0, 0, 0)
[info] epollcat.TcpSuite:
[info] ==> X epollcat.TcpSuite.local and remote addresses 0.01s munit.ComparisonFailException: /Users/armanbilge/code/epollcat/tests/shared/src/test/scala/epollcat/TcpSuite.scala:133
[info] 132:                assertEquals(clientRemote, serverLocal)
[info] 133:                assertEquals(serverRemote, clientLocal)
[info] 134:              }
[info] values are not the same
[info] => Obtained
[info] /<IP_ADDRESS>:63437
[info] => Diff (- obtained, + expected)
[info] -/<IP_ADDRESS>:63437
[info] +/0:0:0:0:0:0:7F00:1:63437
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] 	epollcat.TcpSuite
[error] (testsNative / Test / testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 1 s, completed Nov 13, 2022, 6:09:29 PM

@LeeTibbert if you have a chance to take a look at this as well I would be much obliged. I am a bit at a loss what to do next, besides disabling this test in CI.

@armanbilge I suggest you disable, but not delete, the test whilst I study this.  Sorry that it is wasting your time.
Don't you just love intermittent failures?  Unfortunately, sometimes they are the most informative.
0:0:0:0:0:0:7F00:1 is a valid address, but one which the epollcat software should never see. It is an
IPv4 address, expressed as an IPv6 address, not an IPv6-compatible address. You & I are expecting
the latter (0:0:0:0:0:FFFF:7F00:1) form.  There is a name for the former form, but it is so seldom
seen that I have forgotten it.
I suspect some form of round-robining, possibly in how clients are connecting to the server socket,
or in some getaddrinfo() call.
Anyway, as you surmise, a fix is probably days, not hours away.

By "by never see" I mean that it is used in InetAddress* internally.
By my reading it is the second assert which is failing
:                assertEquals(serverRemote, clientLocal)

if that is true, I need to look clientLocal and the InetAddress equals override.
My plan is to clone the epollcat repository onto my macM1 and see if I can
replicate over 20 or 30 runs.

I've now identified this is a regression in SN 0.4.8. If I take my debug commit https://github.com/armanbilge/epollcat/commit/8afb7bec5399853600d863a1053c626afde9a024 and revert it, I now consistently get this as output.
sbt:root> testsNative/testOnly epollcat.TcpSuite
[info] Starting process '/Users/armanbilge/code/epollcat/tests/native/target/scala-2.13/tests-test-out' on port '65059'.
List(28, 30, -2, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0)
List(28, 30, -2, 37, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0)
List(28, 30, -2, 38, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0)
[info] epollcat.TcpSuite:
[info]   + local and remote addresses 0.01s
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1

0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1 is the IPv6 loopback address "::1"
which may or may not be reasonable, depending upon how the client made the connection (IPv4 or IPv6).
Glad this bug is not a University final exam question.

I am finding that I can avoid having to do a total reboot by "sbt> testsNative/clean"  and then re-running my
test (only the remote & local test). In (native) TcpSuite (i.e. "localhost") it fails the first time and succeeds thereafter.
Tcp6Suite (::1) seems to always work.  All this on macOS 12.6(?).  I have been mainly using Scala 3.2.1 but
fell back to Scala 2.13.10 to see what happens there.

One of the many changes in InetAddress* that happened between SN 0.4.7 and 0.4.8 is that
the resolution of "localhost" may have changed from (IPv6) "::1" (all zero, then 1) to
IPv4 (<IP_ADDRESS>).  This change was to make SN act the same as JVM by using the default "false"
for "java.net.preferIPv6Addresses" (SN Issue #2849).
I am still tracing trying to figure out what is going on here and what should be going on here.

A quick note as I end my sprint.
The reported bug also occurs on macOS 12.6 M1 hardware.  It happens intermittently, almost always as
the first run after the executable starts up.
It looks like the fix in SN 0.4.8 to follow ScalaJVM java behavior in preferring "localhost" as an IPv4mapped IPv6
address "::FFFF:<IP_ADDRESS>" rather than an IPv6 address "0:0:0:0:0:0:0:1", better known as "::1" revealed
a "quirk"  in macOS getsockname().
It appears that macOS getsockname() can and does, especially on the first call after an executable has
started, return an IPv4 compatible IPv6 address (i.e. bytes 10 & 11 are 0 and not FF). The rest of the time
it appears to be consistently returning an IPv4 mapped IPv6 address.  The IPv4 compatible address is not
wrong just unexpected & unusual.
I have not tried on Linux or other operating systems.  I also need to check
if SN javalib uses getsockname().
I have a provisional fix to epollcat SocketHelpers#toInet6SocketAddress which I hope to
submit next sprint.
Even though this appears to be a macOS quirk, my current thinking is to enable the
fix across the board and be done with it.
I had been suspecting accept, especially since slightly different paths are taken on
Linux and macOS, but accept proved itself to work as desired & expected.

Thanks for all your investigating!

It appears that macOS getsockname() can and does, especially on the first call after an executable has
started, return an IPv4 compatible IPv6 address (i.e. bytes 10 & 11 are 0 and not FF). The rest of the time
it appears to be consistently returning an IPv4 mapped IPv6 address. The IPv4 compatible address is not
wrong just unexpected & unusual.

I see. A "quick" seems appropriate indeed. I look forward to seeing your fix :)
Could 0:0:0:0:0:0:7F00:1 be a valid IPv6 address? My concern is whether it is possible to distinguish between when that is an IPv6 address vs an IPv4-compatible IPv6 address.
I am also curious if or how how other libraries work around this quirk, or even the JDK itself.

0:0:0:0:0:0:7F00:1 is a valid IPv6 address and is as valid as any other, in that it is
well described in the RFC.   It is an IPv4 compatible IPv6 address (not "mapped").
There is a rat's nest of applicable and obsoleted RFC's and drafts (which may
not be final or "standard", is the common practice.   https://www.rfc-editor.org/rfc/rfc4291.html
is one I found easily, I am not sure if it is the latest.  Not to be an RFC lawyer, but to return
to original sources, Section 2.5.5 describes:

2.5.5.  IPv6 Addresses with Embedded IPv4 Addresses

   Two types of IPv6 addresses are defined that carry an IPv4 address in
   the low-order 32 bits of the address.  These are the "IPv4-Compatible
   IPv6 address" and the "IPv4-mapped IPv6 address".

<IP_ADDRESS>.  IPv4-Compatible IPv6 Address

   The "IPv4-Compatible IPv6 address" was defined to assist in the IPv6
   transition.  The format of the "IPv4-Compatible IPv6 address" is as
   follows:

   |                80 bits               | 16 |      32 bits        |
   +--------------------------------------+--------------------------+
   |0000..............................0000|0000|    IPv4 address     |
   +--------------------------------------+----+---------------------+

   Note: The IPv4 address used in the "IPv4-Compatible IPv6 address"
   must be a globally-unique IPv4 unicast address.

   The "IPv4-Compatible IPv6 address" is now deprecated because the
   current IPv6 transition mechanisms no longer use these addresses.
   New or updated implementations are not required to support this
   address type.

Section <IP_ADDRESS> goes on the describe the "IPv4-Mapped IPv6 Address" (::FFFF:<IP_ADDRESS>)
address we know, love, & expect.
These days, the ::0000:<IP_ADDRESS> is almost always considered a mistake.
Java 17 does not even mention their handling:
Special IPv6 address

    IPv4-mapped address 	Of the form ::ffff:w.x.y.z, this IPv6 address is used to represent an IPv4 address. It allows the native program to use the same address data structure and also the same socket when communicating with both IPv4 and IPv6 nodes.

    In InetAddress and Inet6Address, it is used for internal representation; it has no functional role. Java will never return an IPv4-mapped address. These classes can take an IPv4-mapped address as input, both in byte array and text representation. However, it will be converted into an IPv4 address.

As I think you demonstrated earlier, one can create them explicitly and they are not forced to
"mapped"
import java.net._

val a = Array(0, 0,0,0,0,0,0,0,0,0,0,0,127,0,0,1).map(x => x.toByte)
val ia6 = Inet6Address.getByAddress(":", a, 0)

val f = ia6.asInstanceOf[Inet6Address].isIPv4CompatibleAddress()
printf(s"f: $f\n")
print(ia6.toString())

If we want to maintain that behavior, and I think we do and I think your last entry
implies that, I can put the 'quirkcorrection close to thequirk source (getsockname()` and
leave the general handling undisturbed.  "Least strong intervention".
What do you think (beyond, "I am glad I never became a network engineer. And I thought async was
bad!")?

0:0:0:0:0:0:7F00:1 is a valid IPv6 address and is as valid as any other, in that it is
well described in the RFC. It is an IPv4 compatible IPv6 address (not "mapped").

Ok, thanks, so if I understand correctly the single unique interpretation of 0:0:0:0:0:0:7F00:1 is as an IPv4 compatible IPv6 address: there is no other interpretation. That's a small relief.
Thanks for quoting the RFC. That was helpful. I think 😅
Your "quick quirk correction" sounds good to me!


Test example fails

Hi,
I was trying to do a test run on the example data mentioned in the repo. While doing so, I get the following error:
(py27) Taraks-MacBook-Pro:deepvariant tarakshisode$ nextflow run main.nf --test
N E X T F L O W  ~  version 18.10.1
Launching `main.nf` [silly_chandrasekhar] - revision: 0f2d95c8e6
[warm up] executor > local
[c2/77de17] Submitted process > preprocessBAM (NA12878_S1.chr20.10_10p1mb.bam)
[d1/175683] Submitted process > preprocessFASTA
[cc/dc2da0] Submitted process > makeExamples (NA12878_S1.chr20.10_10p1mb.bam)
WARN: Unable to stage foreign file: s3://deepvariant-data/models (try 1) -- Cause: Read timed out

WARN: Unable to stage foreign file: s3://deepvariant-data/models (try 2) -- Cause: Read timed out
WARN: Unable to stage foreign file: s3://deepvariant-data/models (try 3) -- Cause: Read timed out
ERROR ~ Error executing process > 'call_variants (NA12878_S1.chr20.10_10p1mb.bam)'

Caused by:
  Can't stage file s3:///deepvariant-data/models -- reason: Read timed out

Command executed:

  /opt/deepvariant/bin/call_variants     --outfile call_variants_output.tfrecord     --examples<EMAIL_ADDRESS>   --checkpoint dv2/models/model.ckpt     --num_readers 8

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /Users/tarakshisode/DeepVariant/work/fc/12d271c326ec52b38d22fa787a8990

Tip: when you have fixed the problem you can continue the execution appending to the nextflow command line the option `-resume`

 -- Check '.nextflow.log' file for details
Oops .. something went wrong

Any idea why?

Hi @tarak77,
Sorry for the late reply.
I'm not exactly sure why you get this error as the model file does exist at s3:///deepvariant-data/models.
If you want to test out DeepVariant in Nextflow I recommend you have a look at nf-core/DeepVariant which uses the newer version of DeepVariant.
If you already have Nextflow and docker installed you can run a test with:
nextflow run nf-core/deepvariant -profile test,docker

If you have any more issues feel free to make them there where I can also reply.
Thanks


Reload user-defined modules during reload()

#2251 didn't work, so here is a new attempt :)
🎈 Motivation
In #2240, I've added a reload() command that will reuse the OpenGL window to speed up reloading (instead of having to quit the active ManimGL Python interpreter session, then starting it again). Unfortunately, I've missed out on the following point from this comment.

In fact, for code changes outside the file of the scene, from what I can tell the reload call does not incorporate those, whereas the command+A, command+R on that file would.

While in theory the reload() command will perform every single step that we also do when entering ManimGL for the very first time (except for parsing arg params), in practice we observe some module caching which this PR addresses.

[!tip]
With regards to the main use case: yes, one could just select the changed code and run it directly in the terminal to update it. But this is not always convenient, e.g. imagine you want to change the code at multiple occurrences (maybe even spread across different files) and see the effects. E.g. consider also my evolute example here. Or the case where you are a Manim developer and want to see the effects directly when you changed multiple files deep down at the heart of Manim (set IGNORE_MANIMLIB_MODULES = False in this case).
The main idea behind reload() is that in situations where a checkpoint_paste() might not work or is cumbersome, instead of having to Ctrl+C the active session, then reloading everything again, we have a reload(<optional_line_number>) command as introduced in #2240. To the user, the final behavior should look exactly like Ctrl+C then spinning up again, with the benefit of the OpenGL window staying open all the time and a lot faster reload times (like milliseconds vs. multiple seconds).

💻 Proposed changes
We introduce a new ModuleLoader. If the user doesn't execute reload(), the get_module() method of that loader is exactly the same as beforehand with the main part being:
module_name = file_name.replace(os.sep, ".").replace(".py", "")
spec = importlib.util.spec_from_file_location(module_name, file_name)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)

as inspired by the Python docs. So good news, we should not break anything when reload() is not executed.
Now to the interesting case with reload(). A first attempt was made in #2251 by deleting module names from the sys.modules to force them being reloaded. This didn't end up well and resulted in error messages for third-party libraries.
In this PR, we use the proper way of importlib.reload(module). But it's unfortunately not as simple as that, so here's an overview of what we do (details in docstrings):

whenever we are in the reload() case...
Execute spec.loader.exec_module(module) with a replaced __import__ function that tracks those imports, such that we know what modules the user imported in their files.
Out of those gathered modules, reload those that are "user-defined", e.g. when custom files are imported, but NOT built-in Python system modules like sys, time etc. and NOT external 3rd-party libraries like numpy etc.
To achieve this reloading across multiple levels, we implement a recursive deep_reload(). I've tried with IPython.lib.deepreload but this doesn't work since also system modules and 3rd-party modules will be imported (and those should not be reloaded).
Note: The IGNORE_MANIMLIB_MODULES controls whether manimlib modules should be ignored. I've set this to True by default since most users won't make any changes to the Manim code itself. Set it to False and the reload() will take a bit longer to reload everything (still a lot quicker than killing ManimGL altogether and reinitializing it).

🧪 Test

See test files
# file a.py
import sys

sys.path.append(".")
from manimlib import *
import numpy as np  # just to have another dependency to test the import

from b import BAndC

class D:
    def answer_d(self):
        return "Hello from D"

class AScene(Scene):
    def construct(self):
        ## Starting
        print("Hello from A")

        ## B
        b_and_c = BAndC()
        print(b_and_c.answer_b())

        ## C
        print(b_and_c.answer_c())

        ## D
        d = D()
        print(d.answer_d())

# file b.py
from c import C

class BAndC:
    def __init__(self):
        self.c = C()

    def answer_b(self):
        return "Hello from B"

    def answer_c(self):
        return self.c.answer_c()

# file c.py
class C:
    def answer_c(self):
        return "Hello from C"

manimgl "/path/to/a.py" AScene -se 29

Run and see the respective Hello from X statements printed to the console.
Change any string (a, b, c or d) and save the file.
Preview via checkpoint_paste(): still see the old strings printed.
reload()
Now see the updated statements in the console and also debug messages of which modules were reloaded 🎉 (e.g. here a, c and b in that order)

🎬 And here two videos to showcase this.
First the example from the PR comment:
https://github.com/user-attachments/assets/ccc154ef-358a-45c1-957b-4dbc7bf41051
An then how this could be beneficial for Manim devs by setting IGNORE_MANIMLIB_MODULES = False.
https://github.com/user-attachments/assets/9ce45145-5289-4fc8-9bc1-678a44087184

This works alright for me now, nice!
I appreciate the contribution, and am happy to merge, though I do feel like I should express some wariness around adding complexity without a strong need. In this case, quitting and restarting is perhaps a little bothersome—a second or two of of waiting in an otherwise smooth workflow can be—but it's right on the edge.
The module_loader.py functionality is flirting with functionality that feels best suited for python itself to manage: Tracking which modules are built in, making a wrapper around builtins.import, etc. I understand how given what you want to do, it makes sense to have all this, and you've done it very cleanly. But in general, I feel like manim has a lot of unnecessary complexity and hackiness in places, and it'd be nice to mitigate that or nip it in the bud when possible.

Thanks a lot for reviewing and merging, this will allow me to include this in the Manim Notebook extension via https://github.com/Manim-Notebook/manim-notebook/pull/95

The module_loader.py functionality is flirting with functionality that feels best suited for python itself to manage: [...]

Yes, indeed, it did indeed feel "hacky" at times and I was a bit disappointed that I couldn't find an easier way to do all this and that I had to dive that deeply into the Python module mechanism. I strongly hope that this will remain my last "hacky" contribution to Manim. I also hope that this PR doesn't introduce any unwanted side-effects down the line; feel free to ping me in an issue should this be the case somewhere in the future. But I feel pretty optimistic that it won't happen given that the code is really just executed when reload() is called.

In this case, quitting and restarting is perhaps a little bothersome—a second or two of of waiting in an otherwise smooth workflow can be—but it's right on the edge.

Fair enough. Where reload() might shine is when you modify multiple places of code at the same time. In this case, you'd have to select all occurrences and run them one after the other (while maybe even considering the order of statements). Another case: when you're working in a bigger class, you are forced to always select the whole class, which in my eyes is tedious ;) That said, I do totally understand the wariness and try to avoid such PRs in the future.


Add CDL concept demo codes

This PR includes the initial integration for CDL concept demo codes into GDP.
The CDL Concept Demo was not yet integrated with the GDP HMI Launcher because it was designed to run different binaries on three GDP devices respectively.
Currently, each GDP device must be tested by running it through a pre-provided script included in the image.
This code will be able to integrate with the HMI Launcher in the future.
[GDP-500] Integrate Car Data Logger (CDL) into GDP

Great.  The build checks suggest there might be some CommonAPI version mismatch here so let's investigate.

@summeromance Could you post the commit hash values that you have used for other layers when you built it successfully?  (Most likely we are using different versions of meta-ivi)

I tested this with morty branch of genivi-dev-platform.
Hash values are

genivi-dev-platform : 79002aeb7da0685fce88e2770d0512e5ac6fca34

meta-ivi : 860f0ce4540412c25d74dd9f53859b24fcb56300
meta-genivi-dev : b5b76411194b2b4a73f9bff2a4ce6b7c41b24904

Ah ok, that explains it.  The build checks executed here are using master, which is not yet switched over to Morty.  So that definitely explains the mismatch in CommonAPI version.  Which hardware(s) did you build for?

I tested on Raspberry Pi3.
Should I create PR to morty branch?

Not requesting you to change the PR right now because Morty branch is temporary and there are no build checks for PRs to that branch anyway so we will not learn anything new.  No I think it's more likely we switch master over to Morty and then rebase this on top of that.  I'll discuss with @zeenix.

We need to discuss about packagegroup because purpose of packagegroup is for component bundle.
However, CDL is just a component and I thought that packagegroup is not required for CDL.
@chbae  I agree with your assessment.  I have too many other more important things to consider now however.  Maybe the rest of you can just agree on something simple?

FYI: On a separate machine I am building this on top of GDP Morty branch as an alternative build check.  I imagine I will have the result tomorrow.

@gunnarx Thanks
@zeenix Please give me your opinion. I would like to use "packagegroup-gdp-extended".

@summeromance
Your PR is not merged and you don't need to create new commit to apply my comment.
Please use "amend" commit in your existed commit.

It seems to be a mistake because I am not familiar with the GitHub process and it is first time amend to the commit that already pushed to remote. Two commits that added for packagegroups have disappeared.
There is a discussion about packagegroup, so is it better to make a new commit if it is determined, or should I re-create the commit as it is?

I'm a bit confused, the latest commit is only adding the recipe but not adding it to the image (which would explain the builds magically starting to succeed). Regarding package-group, I don't have a strong opinion but looking at the existing package-groups, it seems that we would like a -cdl packagegroup, similar to -fsa, -hmi, -browser etc.

I'm a bit confused, the latest commit is only adding the recipe but not adding it to the image (which would explain the builds magically starting to succeed).

@zeenix As I commented, that is my fault.

@zeenix Please give me your opinion. I would like to use "packagegroup-gdp-extended".

Regarding package-group, I don't have a strong opinion but looking at the existing package-groups, it seems that we would like a -cdl packagegroup, similar to -fsa, -hmi, -browser etc.

@zeenix @chbae zeenix suggested creating a cdl package group, and chbae suggested that create an extended package group and include CDL there. It seems difficult to determine which package group the CDL concept demo should be included in.

@summeromance people can't always have the same opinions. I explained that i don't have strong opinions but presented the reasoning behind my opinion. Maybe you can give it a thought and provide your conclusion/assessment?

@summeromance
If you have any plan to add more components in packagegroup-gdp-cdl, It's better to use packagegroup-gdp-cdl. Or If not, please use common packagegroup (packagegroup-gdp-extended). That's one of opinion as @zeenix said.

@zeenix @chbae Thank you for the suggestions.
I mean it's hard to decide which package group to include in the CDL, because the CDL is not yet part of a specific expert group and is being run as a subproject of KREG. This is also why I was worried about where to put the UML when I registered the CDL as a Placeholder Component.
So I agree with chbae, but the code I'm registering now is a kind of proof of concept as Placeholder Component of the CDL, and now I'm implementing the PoC of the abstract component, which I will also promote to integrate into the GDP later, so I agree with zeenix also.

@chbae @zeenix I created the commit to add the package group for the CDL, which is the same as the commit I originally made. Once again thank you for your suggestions.

@summeromance Thanks. Seems the build is now failing:
12:05:36.091 | ../../../git/project/interface/src-gen/v1/com/ivis/CDL/ClientAPILocalDBusStubAdapter.cpp:22:200:   required from here
12:05:36.091 | ../../../git/project/interface/src-gen/v1/com/ivis/CDL/ClientAPILocalDBusStubAdapter.hpp:167:18: error: type 'CommonAPI::DBus::DBusStubAdapter' is not a direct or virtual base of 'v1::com::ivis::CDL::ClientAPILocalDBusStubAdapter<v1::com::ivis::CDL::ClientAPILocalStub>'
12:05:36.091 |              _stub) {
12:05:36.091 |                   ^
12:05:36.091 | In file included from ../../../git/project/interface/src-gen/v1/com/ivis/CDL/ClientAPILocalDBusStubAdapter.hpp:23:0,
12:05:36.091 |                  from ../../../git/project/interface/src-gen/v1/com/ivis/CDL/ClientAPILocalDBusStubAdapter.cpp:11:
12:05:36.091 | /var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/sysroots/dragonboard-410c/usr/include/CommonAPI-3.1/CommonAPI/DBus/DBusStubAdapterHelper.hpp: In instantiation of 'static bool CommonAPI::DBus::DBusStubSignalHelper<In_<CommonAPI::DBus::DBusInputStream, CommonAPI::DBus::DBusOutputStream, InArgs_ ...> >::sendSignal(const DBusStub_&, const char*, const char*, const InArgs_& ...) [with DBusStub_ = v1::com::ivis::CDL::ClientAPILocalDBusStubAdapterInternal<v1::com::ivis::CDL::ClientAPILocalStub>; In_ = CommonAPI::SerializableArguments; InArgs_ = {CommonAPI::Deployable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, CommonAPI::DBus::StringDeployment>, CommonAPI::Deployable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, CommonAPI::DBus::StringDeployment>, CommonAPI::Deployable<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, CommonAPI::DBus::StringDeployment>, CommonAPI::Deployable<CommonAPI::Variant<bool, short int, short unsigned int, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, CommonAPI::DBus::VariantDeployment<CommonAPI::EmptyDeployment, CommonAPI::EmptyDeployment, CommonAPI::EmptyDeployment, CommonAPI::EmptyDeployment, CommonAPI::DBus::StringDeployment> >}]':
12:05:36.091 | ../../../git/project/interface/src-gen/v1/com/ivis/CDL/ClientAPILocalDBusStubAdapter.hpp:139:19:   required from 'void v1::com::ivis::CDL::ClientAPILocalDBusStubAdapterInternal<_Stub, _Stubs>::fireNotifyDataEvent(const string&, const string&, const string&, const CDLData&) [with _Stub = v1::com::ivis::CDL::ClientAPILocalStub; _Stubs = {}; std::__cxx11::string = std::__cxx11::basic_string<char>; v1::com::ivis::CDL::ClientAPILocalTypes::CDLData = CommonAPI::Variant<bool, short int, short unsigned int, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >]'
12:05:36.091 | ../../../git/project/interface/src-gen/v1/com/ivis/CDL/ClientAPILocalDBusStubAdapter.cpp:37:1:   required from here
12:05:36.091 | /var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/sysroots/dragonboard-410c/usr/include/CommonAPI-3.1/CommonAPI/DBus/DBusStubAdapterHelper.hpp:259:26: error: 'const class v1::com::ivis::CDL::ClientAPILocalDBusStubAdapterInternal<v1::com::ivis::CDL::ClientAPILocalStub>' has no member named 'getDBusAddress'
12:05:36.091 |          return(sendSignal(_stub.getDBusAddress().getObjectPath().c_str(),
12:05:36.091 |                           ^
12:05:36.091 | /var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/sysroots/dragonboard-410c/usr/include/CommonAPI-3.1/CommonAPI/DBus/DBusStubAdapterHelper.hpp:259:26: error: 'const class v1::com::ivis::CDL::ClientAPILocalDBusStubAdapterInternal<v1::com::ivis::CDL::ClientAPILocalStub>' has no member named 'getDBusAddress'
12:05:36.092 | /var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/sysroots/dragonboard-410c/usr/include/CommonAPI-3.1/CommonAPI/DBus/DBusStubAdapterHelper.hpp:259:26: error: 'const class v1::com::ivis::CDL::ClientAPILocalDBusStubAdapterInternal<v1::com::ivis::CDL::ClientAPILocalStub>' has no member named 'getDBusConnection'
12:05:36.092 | Makefile:431: recipe for target 'release/arm/.obj/ClientAPILocalDBusStubAdapter.o' failed
12:05:36.092 | make[1]: *** [release/arm/.obj/ClientAPILocalDBusStubAdapter.o] Error 1
12:05:36.092 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerDBusProxy.cpp: In function 'void v1::com::ivis::ModeManager::registerModeManagerDBusProxy()':
12:05:36.092 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerDBusProxy.cpp:30:38: error: 'class CommonAPI::DBus::Factory' has no member named 'registerInterface'
12:05:36.092 |      CommonAPI::DBus::Factory::get()->registerInterface(initializeModeManagerDBusProxy);
12:05:36.092 |                                       ^
12:05:36.092 | Makefile:448: recipe for target 'release/arm/.obj/ModeManagerDBusProxy.o' failed
12:05:36.092 | make[1]: *** [release/arm/.obj/ModeManagerDBusProxy.o] Error 1
12:05:36.092 | In file included from ../../../git/project/mode-manager/src/ModeManagerService.h:27:0,
12:05:36.092 |                  from ../../../git/project/mode-manager/src/ModeManagerService.cpp:23:
12:05:36.092 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerStubDefault.hpp:45:5: error: expected initializer before ':' token
12:05:36.092 |      : public virtual ModeManagerStub {
12:05:36.092 |      ^
12:05:36.092 | In file included from ../../../git/project/mode-manager/src/ModeManagerService.cpp:23:0:
12:05:36.092 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.092 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.092 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.092 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.092 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected declaration before end of line
12:05:36.092 | Makefile:413: recipe for target 'release/arm/.obj/ModeManagerService.o' failed
12:05:36.092 | make[1]: *** [release/arm/.obj/ModeManagerService.o] Error 1
12:05:36.093 | make[1]: Leaving directory '/var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/work/aarch64-poky-linux/cdl-concept-demo/1.0-r0/build/project/cdl-client-api'
12:05:36.093 | Makefile:79: recipe for target 'sub-project-cdl-client-api-make_first' failed
12:05:36.093 | make: *** [sub-project-cdl-client-api-make_first] Error 2
12:05:36.093 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerDBusProxy.cpp: In function 'void v1::com::ivis::ModeManager::registerModeManagerDBusProxy()':
12:05:36.093 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerDBusProxy.cpp:30:38: error: 'class CommonAPI::DBus::Factory' has no member named 'registerInterface'
12:05:36.093 |      CommonAPI::DBus::Factory::get()->registerInterface(initializeModeManagerDBusProxy);
12:05:36.093 |                                       ^
12:05:36.093 | ../../../git/project/interface/src-gen/v1/com/ivis/MediaManager/PlaybackControlDBusProxy.cpp: In function 'void v1::com::ivis::MediaManager::registerPlaybackControlDBusProxy()':
12:05:36.093 | ../../../git/project/interface/src-gen/v1/com/ivis/MediaManager/PlaybackControlDBusProxy.cpp:30:38: error: 'class CommonAPI::DBus::Factory' has no member named 'registerInterface'
12:05:36.093 |      CommonAPI::DBus::Factory::get()->registerInterface(initializePlaybackControlDBusProxy);
12:05:36.093 |                                       ^
12:05:36.093 | Makefile:478: recipe for target 'release/arm/.obj/ModeManagerDBusProxy.o' failed
12:05:36.093 | make[1]: *** [release/arm/.obj/ModeManagerDBusProxy.o] Error 1
12:05:36.093 | In file included from ../../../git/project/media-manager/src/playback/PlaybackService.h:26:0,
12:05:36.093 |                  from ../../../git/project/media-manager/src/playback/AbstractPlayer.cpp:24:
12:05:36.093 | ../../../git/project/interface/src-gen/v1/com/ivis/MediaManager/PlaybackControlStubDefault.hpp:45:5: error: expected initializer before ':' token
12:05:36.093 |      : public virtual PlaybackControlStub {
12:05:36.093 |      ^
12:05:36.093 | ../../../git/project/media-manager/src/playback/AbstractPlayer.cpp:201:1: error: expected '}' at end of input
12:05:36.093 |  }
12:05:36.093 |  ^
12:05:36.093 | ../../../git/project/media-manager/src/playback/AbstractPlayer.cpp:201:1: error: expected '}' at end of input
12:05:36.094 | ../../../git/project/media-manager/src/playback/AbstractPlayer.cpp:201:1: error: expected '}' at end of input
12:05:36.094 | ../../../git/project/media-manager/src/playback/AbstractPlayer.cpp:201:1: error: expected '}' at end of input
12:05:36.094 | Makefile:461: recipe for target 'release/arm/.obj/AbstractPlayer.o' failed
12:05:36.094 | make[1]: *** [release/arm/.obj/AbstractPlayer.o] Error 1
12:05:36.094 | Makefile:466: recipe for target 'release/arm/.obj/PlaybackControlDBusProxy.o' failed
12:05:36.094 | make[1]: *** [release/arm/.obj/PlaybackControlDBusProxy.o] Error 1
12:05:36.094 | In file included from ../../../git/project/mode-manager/src/ModeManagerService.h:27:0,
12:05:36.094 |                  from ../../../git/project/mode-manager/main.cpp:27:
12:05:36.094 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerStubDefault.hpp:45:5: error: expected initializer before ':' token
12:05:36.094 |      : public virtual ModeManagerStub {
12:05:36.094 |      ^
12:05:36.094 | In file included from ../../../git/project/mode-manager/main.cpp:27:0:
12:05:36.094 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.094 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.094 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.094 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.094 | ../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected declaration before end of line
12:05:36.094 | Makefile:385: recipe for target 'release/arm/.obj/main.o' failed
12:05:36.094 | make[1]: *** [release/arm/.obj/main.o] Error 1
12:05:36.094 | In file included from ../../../git/project/media-manager/src/playback/PlaybackService.h:26:0,
12:05:36.094 |                  from ../../../git/project/media-manager/main.cpp:27:
12:05:36.094 | ../../../git/project/interface/src-gen/v1/com/ivis/MediaManager/PlaybackControlStubDefault.hpp:45:5: error: expected initializer before ':' token
12:05:36.095 |      : public virtual PlaybackControlStub {
12:05:36.095 |      ^
12:05:36.095 | ../../../git/project/media-manager/main.cpp:59:1: error: expected '}' at end of input
12:05:36.095 |  }
12:05:36.095 |  ^
12:05:36.095 | ../../../git/project/media-manager/main.cpp:59:1: error: expected '}' at end of input
12:05:36.095 | ../../../git/project/media-manager/main.cpp:59:1: error: expected '}' at end of input
12:05:36.095 | ../../../git/project/media-manager/main.cpp:59:1: error: expected '}' at end of input
12:05:36.095 | Makefile:418: recipe for target 'release/arm/.obj/main.o' failed
12:05:36.095 | make[1]: *** [release/arm/.obj/main.o] Error 1
12:05:36.095 | In file included from release/arm/.moc/../../../../../../git/project/mode-manager/src/ModeManagerService.h:27:0,
12:05:36.095 |                  from release/arm/.moc/moc_ModeManagerService.cpp:9:
12:05:36.095 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerStubDefault.hpp:45:5: error: expected initializer before ':' token
12:05:36.095 |      : public virtual ModeManagerStub {
12:05:36.095 |      ^
12:05:36.095 | In file included from release/arm/.moc/moc_ModeManagerService.cpp:9:0:
12:05:36.095 | release/arm/.moc/../../../../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.095 | release/arm/.moc/../../../../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.095 | release/arm/.moc/../../../../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.095 | release/arm/.moc/../../../../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected '}' before end of line
12:05:36.095 | release/arm/.moc/../../../../../../git/project/mode-manager/src/ModeManagerService.h:34:20: error: expected declaration before end of line
12:05:36.095 | Makefile:416: recipe for target 'release/arm/.obj/moc_ModeManagerService.o' failed
12:05:36.096 | make[1]: *** [release/arm/.obj/moc_ModeManagerService.o] Error 1
12:05:36.096 | make[1]: Leaving directory '/var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/work/aarch64-poky-linux/cdl-concept-demo/1.0-r0/build/project/mode-manager'
12:05:36.096 | Makefile:254: recipe for target 'sub-project-mode-manager-make_first' failed
12:05:36.096 | make: *** [sub-project-mode-manager-make_first] Error 2
12:05:36.096 | In file included from ../../../git/project/av-information-provider/src/AVInformationProvider.h:28:0,
12:05:36.096 |                  from ../../../git/project/av-information-provider/src/AVInformationProvider.cpp:23:
12:05:36.096 | ../../../git/project/interface/src-gen/v1/com/ivis/AVInformationProvider/AVInformationStubDefault.hpp:45:5: error: expected initializer before ':' token
12:05:36.096 |      : public virtual AVInformationStub {
12:05:36.096 |      ^
12:05:36.096 | In file included from ../../../git/project/av-information-provider/src/AVInformationProvider.cpp:23:0:
12:05:36.096 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected '}' before end of line
12:05:36.096 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected '}' before end of line
12:05:36.096 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected '}' before end of line
12:05:36.096 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected '}' before end of line
12:05:36.096 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected declaration before end of line
12:05:36.096 | Makefile:445: recipe for target 'release/arm/.obj/AVInformationProvider.o' failed
12:05:36.096 | make[1]: *** [release/arm/.obj/AVInformationProvider.o] Error 1
12:05:36.096 | In file included from ../../../git/project/cdl-daemon/src/CDLDaemon.h:29:0,
12:05:36.096 |                  from ../../../git/project/cdl-daemon/main.cpp:36:
12:05:36.096 | ../../../git/project/interface/src-gen/v1/com/ivis/CDL/ClientAPILocalStubDefault.hpp:45:5: error: expected initializer before ':' token
12:05:36.096 |      : public virtual ClientAPILocalStub {
12:05:36.096 |      ^
12:05:36.097 | In file included from ../../../git/project/cdl-daemon/main.cpp:37:0:
12:05:36.097 | ../../../git/project/cdl-daemon/src/DBHandler.h:47:20: error: expected '}' before end of line
12:05:36.097 | ../../../git/project/cdl-daemon/src/DBHandler.h:47:20: error: expected '}' before end of line
12:05:36.097 | ../../../git/project/cdl-daemon/src/DBHandler.h:47:20: error: expected '}' before end of line
12:05:36.097 | ../../../git/project/cdl-daemon/src/DBHandler.h:47:20: error: expected '}' before end of line
12:05:36.097 | ../../../git/project/cdl-daemon/src/DBHandler.h:47:20: error: expected declaration before end of line
12:05:36.097 | In file included from ../../../git/project/av-information-provider/src/AVInformationProvider.h:28:0,
12:05:36.097 |                  from ../../../git/project/av-information-provider/main.cpp:24:
12:05:36.097 | ../../../git/project/interface/src-gen/v1/com/ivis/AVInformationProvider/AVInformationStubDefault.hpp:45:5: error: expected initializer before ':' token
12:05:36.097 |      : public virtual AVInformationStub {
12:05:36.097 |      ^
12:05:36.097 | In file included from ../../../git/project/av-information-provider/main.cpp:24:0:
12:05:36.097 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected '}' before end of line
12:05:36.097 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected '}' before end of line
12:05:36.097 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected '}' before end of line
12:05:36.097 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected '}' before end of line
12:05:36.097 | ../../../git/project/av-information-provider/src/AVInformationProvider.h:40:20: error: expected declaration before end of line
12:05:36.097 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerDBusProxy.cpp: In function 'void v1::com::ivis::ModeManager::registerModeManagerDBusProxy()':
12:05:36.097 | ../../../git/project/interface/src-gen/v1/com/ivis/ModeManager/ModeManagerDBusProxy.cpp:30:38: error: 'class CommonAPI::DBus::Factory' has no member named 'registerInterface'
12:05:36.097 |      CommonAPI::DBus::Factory::get()->registerInterface(initializeModeManagerDBusProxy);
12:05:36.097 |                                       ^
12:05:36.097 | Makefile:423: recipe for target 'release/arm/.obj/main.o' failed
12:05:36.098 | make[1]: *** [release/arm/.obj/main.o] Error 1
12:05:36.098 | make[1]: Leaving directory '/var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/work/aarch64-poky-linux/cdl-concept-demo/1.0-r0/build/project/cdl-daemon'
12:05:36.098 | Makefile:279: recipe for target 'sub-project-cdl-daemon-make_first' failed
12:05:36.098 | make: *** [sub-project-cdl-daemon-make_first] Error 2
12:05:36.098 | Makefile:430: recipe for target 'release/arm/.obj/main.o' failed
12:05:36.098 | make[1]: *** [release/arm/.obj/main.o] Error 1
12:05:36.098 | make[1]: Leaving directory '/var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/work/aarch64-poky-linux/cdl-concept-demo/1.0-r0/build/project/av-information-provider'
12:05:36.098 | Makefile:129: recipe for target 'sub-project-av-information-provider-make_first' failed
12:05:36.098 | make: *** [sub-project-av-information-provider-make_first] Error 2
12:05:36.098 | Makefile:702: recipe for target 'release/arm/.obj/ModeManagerDBusProxy.o' failed
12:05:36.098 | make[1]: *** [release/arm/.obj/ModeManagerDBusProxy.o] Error 1
12:05:36.098 | make[1]: *** Waiting for unfinished jobs....
12:05:36.098 | make[1]: Leaving directory '/var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/work/aarch64-poky-linux/cdl-concept-demo/1.0-r0/build/project/av-hmi-application'
12:05:36.098 | Makefile:204: recipe for target 'sub-project-av-hmi-application-make_first' failed
12:05:36.098 | make: *** [sub-project-av-hmi-application-make_first] Error 2
12:05:36.098 | make[1]: Leaving directory '/var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/work/aarch64-poky-linux/cdl-concept-demo/1.0-r0/build/project/media-manager'
12:05:36.098 | Makefile:229: recipe for target 'sub-project-media-manager-make_first' failed
12:05:36.098 | make: *** [sub-project-media-manager-make_first] Error 2
12:05:36.098 | WARNING: exit code 1 from a shell command.
12:05:36.098 | ERROR: oe_runmake failed
12:05:36.098 | ERROR: Function failed: do_compile (log file is located at /var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/tmp/work/aarch64-poky-linux/cdl-concept-demo/1.0-r0/temp/log.do_compile.24908)
12:05:36.098 NOTE: recipe cdl-concept-demo-1.0-r0: task do_compile: Failed
12:05:36.099 NOTE: Running task 4561 of 4755 (ID: 2004, /var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/../meta-genivi-dev/meta-genivi-dev/recipes-dev-hmi/genivi-dev-platform-hmi/qml-example_1.0.bb, do_compile)
12:05:36.224 NOTE: recipe qml-example-1.0-r0: task do_compile: Started
12:05:36.629 NOTE: recipe qtgraphicaleffects-5.6.0+gitAUTOINC+6523d7c484-r0: task do_compile: Succeeded
12:05:55.475 NOTE: recipe gdp-hmi-app-metadata-1.0-r0: task do_compile: Succeeded
12:05:59.805 NOTE: recipe qml-example-1.0-r0: task do_compile: Succeeded
12:08:36.876 NOTE: recipe qtsensors-5.6.0+gitAUTOINC+0b00ee6f6c-r0: task do_compile: Succeeded
12:12:31.569 NOTE: recipe qtquickcontrols-5.6.0+gitAUTOINC+01e52200e2-r0: task do_compile: Succeeded
12:12:38.190 NOTE: recipe qtwayland-5.6.0+gitAUTOINC+26e27845a4-r0: task do_compile: Succeeded
12:12:38.835 NOTE: recipe qtmultimedia-5.6.0+gitAUTOINC+2170a3b0fc-r0: task do_compile: Succeeded
12:12:54.383 NOTE: recipe qtdeclarative-5.6.0+gitAUTOINC+26ff8f9029-r0: task do_package: Succeeded
12:12:54.766 NOTE: Tasks Summary: Attempted 4561 tasks of which 3248 didn't need to be rerun and 1 failed.
12:12:54.767
12:12:54.767 Summary: 1 task failed:
12:12:54.767   /var/lib/go-agent/pipelines/GDP-Yocto-Dragonboard-meta-genivi-dev-GITHUB-PR/gdp/gdp-src-build/../meta-genivi-dev/meta-genivi-dev/recipes-cdl/cdl-concept-demo/cdl-concept-demo.bb, do_compile
12:12:54.767 Summary: There was 1 WARNING message shown.
12:12:54.767 Summary: There were 2 ERROR messages shown, returning a non-zero exit code.
12:12:54.842 [go] Current job status: failed.

12:12:54.842 [go] Start to execute task: <exec command="bash" >
<arg>-c</arg>
<arg>rm -f logs.tar logs.tar.gz && find gdp/gdp-src-build/tmp/work \( -name "*.log" -o -name "log.*" -o -name "run.*" \) -print0 | xargs -0 tar uf logs.tar && gzip logs.tar</arg>
</exec>.
12:13:18.031 [go] Current job status: failed.

The build errors look like the same as before and it is explained by the 5th comment in this thread.  I will kick off a build based on Morty and report.

Initial results for Morty are good or at least consistent with other Morty builds.  I had an unrelated failure on one hardware and still need to rebuild once more before confirming.

My builds on Morty were finally confirmed.     Normal PR-checks should now take over.
(Note: because you opened a pull request, GitHub allows pushing updates to that pull request.  And since you opened the PR from your master branch it means I could actually push into your master branch.  Kind of scary... hope it's not a problem! :-) )

It looks like 9a29db3 was brought in by this PR (it is the hash it has now after rebase) and it seems to be the same purpose as commit 250ee4a2f978 which is already on master.  But they are only almost identical.
Somewhere during recent merges (morty I guess) it seems we had a merge of README changes because now there is also conflicting information on which layer dependency is the right one.   The Renesas section appears twice.  It's logical if changes were made in different parts of the file that a git-merge might cause both sections to be included.  I tried a quick "git blame" and it shows me a mix of these two commits originally from @slawr, plus strangely enough some line changes that git identifies as coming from @chbae...  I think I'm being fooled a bit here.
Anyway I can't easily figure it out and I don't want to waste time.  Honestly I think the easiest thing is to ignore that for this PR and to make a new commit where the layer dependencies are corrected.  (But as I've said before, I think this info should not be here - we would be much better off referring people to the GDP submodules as the single source of dependency information and not waste time on this.  But that's OK if we have different opinion, just @chbae @slawr I think you disagreed with it last time, so would one of you then use your time to fix the conflicting information now - thanks)

Build checks have now executed on the standard master PR checks also.    For minnowboard it was successful in my previous build on Morty.
(finally) +1 from me for merging.
@chbae, for good measure you might want to accept/dismiss your review if you are OK with the changes?

@summeromance Please see my review and others are ok for me.

@chbae Thank you for review. I updated what you reviewed


4.12.1 warnings in console

Since the last release : 4.12.1 we get warnings from v5 i guess
Material-UI: the createMuiTheme function was renamed to createTheme.
The prop justify of ForwardRef(Grid) is deprecated. Use justifyContent instead, the prop was renamed.
And a lot of more ... for each component that evolves in v5

[x] The issue is present in the latest release.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Steps to Reproduce 🕹
You can check console from website example here :
https://codesandbox.io/s/rbtj4
https://codesandbox.io/s/t4hze (same for labs x-grid / datagrid etc..)

Thanks for the feedback. 4.12 includes deprecation warnings relevant to the v5 upgrade. If you don't intend to upgrade to v5 then I suggest rolling back to 4.11.


Target profile page rewrite: Create Protein Interactions summary widget

Use Cases

As a user of the Platform, I want to see how many protein-protein interactions, pathway interactions, and/or enzyme-substrate interactions that have been identified for my protein of interest.

Summary Widget Views

Full-size version
Design and Interaction Notes

For targets where there is data on all three types of interactions (e.g. ESR1), please use the following text in Open Targets Grey - #5a5f5f:

X protein-protein interactions
X pathway interactions
X enzyme-substrate interactions
Please replace X with the relevant number from the GraphQL response.
Also, please colour the icon in Open Targets Grey - #5a5f5f.

For targets where there is data on some, but not all, types of interactions (e.g. PI3 or SPTLC2), please display the interactions with data in Open Targets Grey - #5a5f5f as per point 1 above. For the type(s) of interactions without data, please display the following text in Open Targets Light Grey - #e2dfdf:

0 TYPE-OF-INTERACTION interactions (e.g. 0 protein-protein interactions)
Also, please colour the icon in Open Targets Grey - #5a5f5f.

For targets with no data (e.g. HOTAIR or NUDT1), please use the following text in Open Targets Light Grey - #e2dfdf:

0 protein-protein interactions
0 pathway interactions
0 enzyme-substrate interactions
Also, please colour the icon and the widget container box outline in Open Targets Light Grey - #e2dfdf.

When a user hovers over the summary widget and the target has either a small molecule and/or an antibody tractability assessment, please show the pointer icon and change the box outline to Open Targets Purple - #7b196a. This will provide users with a visual cue that the summary widget is clickable. For more information, please see issue #429.

Design Assets

Protein interactions icon
Open Targets brand palette colour codes

For some genes (e.g. PI3), the widget is incorrectly greyed out, despite having data - see below:

Can we please fix? Once fixed, it's good to go! 👍

Thanks Andrew! Do we know why is this happening?
The inside of the widget has some strange behaviors that we will have to fix when the new network comes into place.

I'm not sure @d0choa, but I think we may need to tweak the logic for the widget. In the current Platform PI3 target profile page, it lists one enzyme-substrate interaction. I've looked at other targets where there is 1 or 2 types of interactions (e.g. SPTLC2) and the logic looks good, so may just be a small fix.

@mirandaio, can we please also update the label to pathway interactions - an o is missing in interactions?

@andrewhercules should be fixed now.

Thanks @mirandaio - looks good - will close this ticket


chore: remove CI on node v13.x

High Level Overview of Change
Title says it all. While I was here I renamed the job to be more descriptive.
Context of Change
We don't use node 13.x anymore. It's also deprecated and we shouldn't be using it anymore.
Type of Change

[x] Tests (You added tests for code that already exists, or your new feature included in this PR)

TypeScript/Hooks Update
N/A
Before / After
Test Plan
CI passes.

Would it make sense to add Node.js 16.x or 17.x now?

Yes, but that was failing (you can see the results in the commit history of this PR) and it wasn't worth the time to figure out what was going wrong right now.


Fetch block number when forking latest block

Partially solves #795
Introduced changes

ForkStateReader converts BlockId::Tag(Latest) to the BlockId::Number(...)

Breaking changes

Checklist

[x] Linked relevant issue
[ ] Updated relevant documentation
[x] Added relevant tests
[x] Performed self-review of the code
[ ] Added changes to CHANGELOG.md

@Arcticae I think we want to avoid spawning random runtimes in forge code

Also, this will save the cache in the Latest file, instead of a block number, we should probably fix that


[bug]: incorrect lagrange evaluation when zeta is roots of unity

Thanks to Shresth and CommonPrefix team for catching this bug (for an audit of the corresponding plonk verifier contract).
Currently the logic for computing lagrange evaluation is wrong when random challenge zeta is roots of unity (which results in vanishing_eval to be zero. In this case, $\mathcal{L}_1(\zeta) = 1$ if $\zeta$ is the first roots-of-unity, and zero otherwise. but our current code would simply panic due to division by zero.
    pub(crate) fn evaluate_lagrange_1_and_n(
        &self,
        zeta: &E::ScalarField,
        vanish_eval: &E::ScalarField,
    ) -> (E::ScalarField, E::ScalarField) {
        let divisor =
            E::ScalarField::from(self.domain.size() as u32) * (*zeta - E::ScalarField::one());
        let lagrange_1_eval = *vanish_eval / divisor;
        let divisor =
            E::ScalarField::from(self.domain.size() as u32) * (*zeta - self.domain.group_gen_inv);
        let lagrange_n_eval = *vanish_eval * self.domain.group_gen_inv / divisor;
        (lagrange_1_eval, lagrange_n_eval)
    }

this also affects evaluate_pi_poly() that internally uses lagrange_i_eval.
We should further double check and fix prover-side code.
Note: we didn't catch this in test because:

we didn't have property test on lagrange poly eval
zeta is random challenge, the probability of it being one of roots of unity is negligible

Thus this issue should also aim to improve tests to guard against this bug
Note2: this is a hard-to-exploit bug (unless our FS is also broken), so current downstream relying on jellyfish shouldn't panic, but the PR that fix this bug should issue a new release and recommend all downstream to migrate over soon.
cc @mrain @chancharles92

comments needed,
I wanted to directly replace our usage with arkworks' API, unfortunately, they only have EvalDomain::evaluate_all_lagrange_coefficients()
maybe I'll just mimic write a fn evaluate_lagrange_coeff_at(domain) for now, and submit a PR upstream for future arkworks to include this API directly in struct EvalDomain, okay?


Error: ext509: malformed Anchor Certificate Extension

Are there any additional details you would like to share?

Command: anchor lcl
Executable: C:\Program Files\Anchor CLI\anchor.exe
Version: 0.1.7 (windows/amd64) Commit: e75e108300982a9bef8d4cc465639591335fb658 BuildDate: 2024-11-12T19:34:37Z
Arguments: []
Flags: []
Timestamp: 2024-11-16T19:57:52.2309958Z
Stdout:
| Let's set up fast and totally free lcl.host HTTPS!

# Setup lcl.host HTTPS Local Development Environment `anchor lcl`
    | We'll set you up to use HTTPS locally in your browsers and other programs.

# Audit lcl.host HTTPS Local Development Environment `anchor lcl audit`
    | We'll compare your local development CA certificates from Anchor and your local trust stores.
    * Comparing local and expected CA certificates…⠋

@sufyanazeem Hey, sorry we didn't get back to you sooner. Are you still seeing this issue, or did it resolve somehow?

It's resolved now .
Thanks for asking
On Wed, Nov 20, 2024 at 10:28 PM Wesley Beary @.***>
wrote:

@sufyanazeem https://github.com/sufyanazeem Hey, sorry we didn't get
back to you sooner. Are you still seeing this issue, or did it resolve
somehow?
—
Reply to this email directly, view it on GitHub
https://github.com/anchordotdev/cli/issues/47#issuecomment-2489175172,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AVE2GLKFW4MVCCSGVWZ5Q632BTBD3AVCNFSM6AAAAABR5EJMNOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBZGE3TKMJXGI
.
You are receiving this because you were mentioned.Message ID:
@.***>

Great, glad to hear it. Just let us know if you have any further questions or feedback. Thanks.


Prevent default action on a link makes a browser ignore it

I want to prevent a browser's default action on an external hyperlink, which is normally done by onclick={return false;}. However, adding
  link.onclick = () => {
      return false;
  };

to LinkAnnotationElement.render() makes the link to be completely ignored by the browser when clicked normally, with the left mouse button. The link only works if "open in new tab", but opening any new page is exactly what I want to disable.
I suspected some security measures, but the link is ignored with http, https, absolute and relative links. Any help?

This works:
  link.onclick = () => {
      fetch(link.href);
      return false;
  };


[HUDI-3475] Initialize hudi table management module

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.

What is the purpose of the pull request
(For example: This pull request adds quick-start document.)
Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@yuzhaojing - I have given @prasannarajaperumal full context on this. and he will take over this. and I ll check-in/chime in as well

@yuzhaojing - I have given @prasannarajaperumal full context on this. and he will take over this. and I ll check-in/chime in as well as needed

Ok, will communicate with him.

@prasannarajaperumal Can you have a look for this pr?


docs: update who is using

Summary
Update who is using:

Add Alibaba and TikTok, see https://github.com/web-infra-dev/rspack/discussions/3170
Unify SVG logo color to black.

Checklist

[ ] Tests updated (or not required).
[ ] Documentation updated (or not required).

✅ Deploy Preview for rspack canceled.

It seems that there is something wrong with netlify.


Math helper pass variable not a number

I am using https://www.npmjs.com/package/handlebars-helpers Math helpers work perfectly, but I just don't know how to use it to work with variable. For example:
{{minus 5 1}}
Output is: 4

But how can i make
{{minus page 1}}
Because it gives me error that first parameter must be a number Thank for any advice

Just mistake sorry variable name is current not page. It works :)


ci: update deploy notification

Description

ci update to deploy notification

Testing required outside of automated testing?

[x] Not Applicable

Screenshots (if appropriate):

[x] Not Applicable

Rollback / Rollforward Procedure

[x] Roll Forward
[ ] Roll Back

Reviewer Checklist

[ ] Description of Change
[ ] Description of outside testing if applicable.
[ ] Description of Roll Forward / Backward Procedure
[ ] Documentation updated for Change

:tada: This PR is included in version 1.203.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Recursive macros

I might be missing something, but a simple recursive macro call
(macro test (lambda () (test)))
(test)

leads to the error
/home/lennu/code/lci/node_modules/eslisp/lib/cli.js:103
      throw err;
      ^

ReferenceError: Error evaluating macro `test` (called at line 27, column 0): test is not defined
    at ctor$.eval (eval at <anonymous> (/home/lennu/code/lci/node_modules/eslisp/lib/built-in-macros.js:610:16), <anonymous>:2:5)
    at /home/lennu/code/lci/node_modules/eslisp/lib/compile.js:184:35
    at listToEstree (/home/lennu/code/lci/node_modules/eslisp/lib/compile.js:191:6)
    at astToEstree (/home/lennu/code/lci/node_modules/eslisp/lib/compile.js:250:25)
    at /home/lennu/code/lci/node_modules/eslisp/lib/translate.js:35:14
    at /home/lennu/code/lci/node_modules/prelude-ls/lib/List.js:158:21
    at /home/lennu/code/lci/node_modules/prelude-ls/lib/List.js:161:4
    at /home/lennu/code/lci/node_modules/prelude-ls/lib/List.js:665:42
    at module.exports (/home/lennu/code/lci/node_modules/eslisp/lib/translate.js:36:7)
    at toEstree (/home/lennu/code/lci/node_modules/eslisp/lib/index.js:9:10)

Calling arguments.callee works, however it is neither recommended or elegant. Is there any alternative?

A macro that calls itself as part of its definition like that would always recurse infinitely: To compile the function that implements the macro test, the compiler would need to know what code (test) returns, so it would need to call the macro test to find out, so it would need to compile the function that implements the macro test, et cetera, recurring infinitely.
What you can do instead is have the macro return code that calls it:
(macro test
 (lambda (x)
  ; If we got an argument, return it.
  ; If we got NO argument, return a call to this macro, with a default.
  (if x (return x)
        (return '(test defaultArgument)))))

(test)
(test notTheDefault)

Output JS:
defaultArgument;
notTheDefault;

It works because the macro isn't called in its definition, but it instead returns code that calls it.

Or if the macro requires fancier recursive computation on its arguments, it might be easier to just define a recursive function inside the macro definition (like factorial here), and call that:
(macro !
 (lambda (thing)

  (var factorial
   (lambda (x)
    (if (== x 1)
     (return x)
     (return (* x (factorial (- x 1)))))))

  (return ((. this atom) (factorial ((. this evaluate) thing))))))

(! 5)
(! (+ 3 2))

Output JS:
120;
120;

Does that answer your question?

Apologies for my simplified example. Of course it would cause an infinite loop if it worked, my intention was to just recreate the error message.
However, you managed to answer the actual question in hand. Thank you!


Move browsers to cross-tool config

Hi. I moved browsers from Autoprefixer’s inner option to browserslist in package.json.
Reason

Autoprefixer team recommends only package.browserslist as a way to set target browsers.
package.browserslist is used in many different tools. It is used by autoprefixer, stylelint-no-unsupported-browser-features, eslint-plugin-compat, postcss-cssnext, postcss-preset-env, postcss-normalize. The most important, this config will be used in next babel-preset-env 7.0.
It is used by autoprefixer-info CLI tool. This CLI tool is part of autoprefixer package, but didn’t work correctly without Browserslist config.
This config is useful not only for tools. The new developer will have the single place to check target browsers in a new project.
CSS-Tricks agree that Browserslist config is really good idea.

Test

Create a new project.
Check project’s package.json for browserslist key.

@gaearon @ai is this still valid for 2?

I'll add it to the final release documentation checklist.


WIP: Adding a 9-Slice pimitive

Totally forgot that I started doing this - in fact I think I started doing this and then decided I needed Ultraviolet to make it work. :sweat_smile:
Anyway, relates to issues #425 and #455.

Closing, I think I'll just start again some time. :laughing:


Publisher: Windows reduce command window pop-ups during Publishing

Changelog Description
Reduce the command line pop-ups that show on Windows during publishing.
Additional info
This PR can be considered a continuation of PR #4481
Also mentioned on discord here
Examples of the pop-ups can be found in the comments here:
https://user-images.githubusercontent.com/49758407/226671061-7a5b578d-5975-4344-827f-2f993bcf3337.mp4
The above video is from BEFORE this PR

The pop-ups:

[x] This PR removes the pop-up shown when opening the Publisher first time
[x] This PR removes one of the pop-up windows shown during extracting/integrating reviews.
[x] This PR fixes this Extract Review call window pop-up
[x] This PR fixes this Extract Burnin call window pop-up

I'm not entirely sure how to resolve the last two. It shows both when running from source and also when running from a openpype_gui.exe in a build.
Testing notes:

Publish away with reviews and the (new?) publisher

This PR fixes this Extract Review call window pop-up
This PR fixes this Extract Burnin call window pop-up

Apparently these were due to calls with shell=True which actually start showing a command prompt again as soon you pass the creationflags we were passing. (shell=True by itself already tries to hide the console that's to be started)
Anyway, with this PR all command line popups in my tests were gone when running from a build.
NOTE: However, when running from source code or using openpype_console a console seems to still pop-up for the run_openpype_process call in Extract Burnin which seems to pop-up a "Python" window. Likely because its triggering the sys.executable (python.exe or openpype_console.exe) which themselves open a console.

I were just about to say LGTM but you merged it before I could press comment 😅


[Feature Request] Add public API to texture packer so it can be used by external scripts interfacing with poi

Add public API to texture packer so it can be used by external scripts, for example when translating shaders I'd want to repack some textures.

What kind of API would you like? What are the parameters the function should have and what are the outputs?
I could simply flag this function as API. It's the same function that the inline packer uses, but it needs lots of detailed parameters to know what to do. `public static Texture2D Pack(TextureSource[] sources, OutputConfig[] outputConfigs, IEnumerable connections, FilterMode targetFilterMode, ColorSpace targetColorSpace, ImageAdjust colorAdjust = null,
float[] kernelX = null, float[] kernelY = null, int kernelLoops = 1, float kernelStrength = 1, bool kernelTwoPass = false, bool kernelGrayscale = false, bool[] kernelChannels = null, bool alphaIsTransparency = false)
Or were you thinking something more simplified?

I really just need a simple way to pack textures into an output texture. So a way to feed in source textures and channels to assign them to. Don't really need any of the other functionality at the moment.

Yeah, well that's what the function does.
Do you just need to pack 4 textures into one? So like a function with just four texture inputs (one/channel)?

Well, I don't really know exactly what I'll need in the future but for now yeah, just packing 2-4 into textures into 1.
We'll use this when translating shaders like standard where repacking textures might be needed.


I don't get any predictions after training model

Hi,
I've been trying to train a model with just 1 class, using the okutama dataset, but at inference time the model predicts nothing, even with a confidence of 0.1.
At training, the loss wasn't bad, 2.8 in train dataset and 4.8 in validation. I don't know why i'm not getting any predictions at inference

@diegoxfx me too, I used the SSD300 to train one 12 classes own dataset, after trianing , I got nothing when run inference SSD 300 with my model.
have you sovled it ?

@Ai-is-light No, i couldn't solve it

Hi, even I  tried to train for one class(products on supermarket shelves) after 80 epochs with 1000 steps per epoch with batch size of 5 and with 300*300 dimensions of images loss was only improved to 1.11,and model was predict random boxes all over testing  images and training image.


Crash on world creation/entering

Minecraft version
1.20.1
Describe the bug
Crash upon entering or creating a world. I have NO IDEA from where did this come from. I haven't touched my modlist or whatever but today when I started my modpack as usual I couldn't create or enter any world that the game would crash. I took a look to the crash text but really couldn't understand much of it. The log file refused to load, it was tooking like 10 minutes to create so I closed the game. I hope we can fix this, thanks!
Logs
https://gist.github.com/PulcioPappi/a64696905dcf8e520f8160c59007ce77
Additional context
Most of the file's mass are the registered advancements since I have the Custom Advancements mod

Nvm delete this I found out and it wasn't even relevant 💀


Work out view authentication/permission options.

Currently, the uploaded views are private, and only viewable when the user explicitly makes them public or adds a bucket policy that suits them. Adding some options around this would make life a lot easier for the user.

Fixed with #4, but wouldn't mind more options, so I'm keeping this open for now


Python sample does not build

After following the steps to build the python sample, I get these build errors.
~/iotedgedev/testpyedge/testPyEdgeSolution$ docker build  --rm -f "/home/testuser1/iotedgedev/testpyedge/testPyEdgeSolution/modules/TestPyModule/Dockerfile.amd64" -t edtrpcontainerreg.azurecr.io/testpymodule:0.0.1-amd64 "/home/testuser1/iotedgedev/testpyedge/testPyEdgeSolution/modules/TestPyModule" && docker push edtrpcontainerreg.azurecr.io/testpymodule:0.0.1-amd64
Sending build context to Docker daemon  14.85kB
Step 1/6 : FROM amd64/python:3.7-slim-buster
 ---> ec68977d0bfc
Step 2/6 : WORKDIR /app
 ---> Using cache
 ---> 9f2a47d10b19
Step 3/6 : COPY requirements.txt ./
 ---> Using cache
 ---> 2c448a56e2b5
Step 4/6 : RUN pip install -r requirements.txt
 ---> Running in 55d7c1bbff45
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7ff32f1f6dd0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/azure-iot-device/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7ff32f33a490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/azure-iot-device/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7ff32f33a7d0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/azure-iot-device/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7ff32f33ab10>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/azure-iot-device/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7ff32f33ae50>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/azure-iot-device/
ERROR: Could not find a version that satisfies the requirement azure-iot-device~=2.7.0 (from versions: none)
ERROR: No matching distribution found for azure-iot-device~=2.7.0
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b251621c-bcb0-b9ce-bd87-90499766137b
Version Independent ID: b23ad340-bdfa-0597-9a0a-51c5e7b18b48
Content: Tutorial - Custom Python module tutorial using Azure IoT Edge
Content Source: articles/iot-edge/tutorial-python-module.md
Service: iot-edge
GitHub Login: @kgremban
Microsoft Alias: kgremban

@ravipath Thanks for the feedback! We are currently investigating and will update you shortly.

@AshokPeddakotla-MSFT I tried a solution from another similar issue that was reported (for Node.js sample) and it worked for me. This was fixed by placing daemon.json at /etc/docker with a dns server information:
{
    "dns": ["<IP_ADDRESS>", "<IP_ADDRESS>"]
}

@ravipath Glad to hear your issue is resolved. Appreciate for sharing the resolution.
We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Install and runtime problem

Hello,
I've spent many hours to get this great work running on my PC (R5 2600, 16GB ram, rtx 3070).
When using cuda 12.1 with Visual Studio 2022 and Torch 2.1 I was able to install all the dependencies and run generating, but it wasn't saving due to "torch not found" error. With cuda 11.6/7 (tried both) and appropriate torch (1.13+cu...) and Visual Studio 2022 I got "compiler not found" error when installing diff-gaussian-rasterization.
Could you please share your full configuration (versions of Python, CUDA, Torch, Visual Studio (if you even use it) and packages listed in requirements.txt)?
EDIT: I reinstalled CUDA 12.1 and when running main.py I'm getting the following error:
[6/6] "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\Hostx64\x64/link.exe" common.o glutil.o rasterize_gl.o torch_bindings_gl.o torch_rasterize_gl.o /nologo /DLL /LIBPATH:F:\DreamGaussian\DG\lib\site-packages\nvdiffrast\torch..\lib /DEFAULTLIB:gdi32 /DEFAULTLIB:opengl32 /DEFAULTLIB:user32 /DEFAULTLIB:setgpu c10.lib c10_cuda.lib torch_cpu.lib torch_cuda.lib -INCLUDE:?warp_size@cuda@at@@YAHXZ torch.lib /LIBPATH:F:\DreamGaussian\DG\lib\site-packages\torch\lib torch_python.lib /LIBPATH:F:\DreamGaussian\DG\Scripts\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\lib\x64" cudart.lib /out:nvdiffrast_plugin_gl.pyd
FAILED: nvdiffrast_plugin_gl.pyd
"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\Hostx64\x64/link.exe" common.o glutil.o rasterize_gl.o torch_bindings_gl.o torch_rasterize_gl.o /nologo /DLL /LIBPATH:F:\DreamGaussian\DG\lib\site-packages\nvdiffrast\torch..\lib /DEFAULTLIB:gdi32 /DEFAULTLIB:opengl32 /DEFAULTLIB:user32 /DEFAULTLIB:setgpu c10.lib c10_cuda.lib torch_cpu.lib torch_cuda.lib -INCLUDE:?warp_size@cuda@at@@YAHXZ torch.lib /LIBPATH:F:\DreamGaussian\DG\lib\site-packages\torch\lib torch_python.lib /LIBPATH:F:\DreamGaussian\DG\Scripts\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\lib\x64" cudart.lib /out:nvdiffrast_plugin_gl.pyd
LINK : fatal error LNK1104: can't open file "python310.lib"
ninja: build stopped: subcommand failed.

@alekolo7 Hi, you'll have to make sure the environment of python + MSVC + CUDA version compatible for Windows, which can be tricky. I recommend using WSL for a try.

Hello, I've spent many hours to get this great work running on my PC (R5 2600, 16GB ram, rtx 3070).
When using cuda 12.1 with Visual Studio 2022 and Torch 2.1 I was able to install all the dependencies and run generating, but it wasn't saving due to "torch not found" error. With cuda 11.6/7 (tried both) and appropriate torch (1.13+cu...) and Visual Studio 2022 I got "compiler not found" error when installing diff-gaussian-rasterization.
Could you please share your full configuration (versions of Python, CUDA, Torch, Visual Studio (if you even use it) and packages listed in requirements.txt)?
EDIT: I reinstalled CUDA 12.1 and when running main.py I'm getting the following error:
[6/6] "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\Hostx64\x64/link.exe" common.o glutil.o rasterize_gl.o torch_bindings_gl.o torch_rasterize_gl.o /nologo /DLL /LIBPATH:F:\DreamGaussian\DG\lib\site-packages\nvdiffrast\torch..\lib /DEFAULTLIB:gdi32 /DEFAULTLIB:opengl32 /DEFAULTLIB:user32 /DEFAULTLIB:setgpu c10.lib c10_cuda.lib torch_cpu.lib torch_cuda.lib -INCLUDE:?warp_size@cuda@at@@yahxz torch.lib /LIBPATH:F:\DreamGaussian\DG\lib\site-packages\torch\lib torch_python.lib /LIBPATH:F:\DreamGaussian\DG\Scripts\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\lib\x64" cudart.lib /out:nvdiffrast_plugin_gl.pyd FAILED: nvdiffrast_plugin_gl.pyd "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\Hostx64\x64/link.exe" common.o glutil.o rasterize_gl.o torch_bindings_gl.o torch_rasterize_gl.o /nologo /DLL /LIBPATH:F:\DreamGaussian\DG\lib\site-packages\nvdiffrast\torch..\lib /DEFAULTLIB:gdi32 /DEFAULTLIB:opengl32 /DEFAULTLIB:user32 /DEFAULTLIB:setgpu c10.lib c10_cuda.lib torch_cpu.lib torch_cuda.lib -INCLUDE:?warp_size@cuda@at@@yahxz torch.lib /LIBPATH:F:\DreamGaussian\DG\lib\site-packages\torch\lib torch_python.lib /LIBPATH:F:\DreamGaussian\DG\Scripts\libs "/LIBPATH:C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\lib\x64" cudart.lib /out:nvdiffrast_plugin_gl.pyd LINK : fatal error LNK1104: can't open file "python310.lib" ninja: build stopped: subcommand failed.

The reason is that the program doesn't know your cuda path！
You can try: export CUDA_HOME=///cuda12.1(your cuda path)

https://github.com/rosinality/stylegan2-pytorch/issues/325#issuecomment-1192107558
check the above for a possible solution, worked for me

Wow, that solves the error. Thank you very much!


[Doc]Fix typo and language issues in the faq.md file

Motivation
Fix typo and language issues in the faq.md file. I'm trying to make it clear, concise and logical.
Modifications

Refine typos;
Refine from the language perspective, make it easy to read and understand;
Refine some answers, make it clear and correct;
Try to re-categorize question types (not finished yet).

@sijie @jiazhai could you please help review when you're available?
and please refine the question in Line 187 (How can I prevent an inactive topic to be deleted under any circumstance? I do not want to set time or space limit for a certain namespace.)

@Jennifer88huang I am closing this file. Since this doesn't seem to be needed any more.

Ok.


Fix string interpolation syntax in C# example for Output.All

The string interpolation syntax for the C# example was using back-tick string interpolation syntax instead of $"{}" interpolation syntax.

@imercerwillow thanks for submitting a pr to fix this up! are you open to accepting @Frassle's suggestions? if so, once those are fixed up we can merge this.


Update Lewes visit slots

Remove Monday and Wednesday morning slots
Remove Friday afternoon slot

Tested in preprod - passed.