panic: runtime error: invalid memory address or nil pointer dereference

I am making a gRPC client to  register a new client .And I use the function registerUser in eca_test.go file . The code shows as below:
Description
//create req
req := &pb.RegisterUserReq{
Id:          &pb.Identity{Id: user.enrollID},
Role:        pb.Role(user.role),
Account:     user.affiliation,
Affiliation: user.affiliationRole,
Registrar: &pb.Registrar{
Id:            &pb.Identity{Id: registrar.enrollID},
Roles:         user.registrarRoles,
DelegateRoles: user.registrarDelegateRoles,
},
Sig: nil}
log.Print(req)
//sign the req
hash := primitives.NewHash()
raw, _ := proto.Marshal(req)
hash.Write(raw)

r, s, err := ecdsa.Sign(rand.Reader, registrar.enrollPrivKey, hash.Sum(nil))
if err != nil {
	msg := "Failed to register user. Error (ECDSA) signing request: " + err.Error()
	return errors.New(msg)
}
R, _ := r.MarshalText()
S, _ := s.MarshalText()
req.Sig = &pb.Signature{Type: pb.CryptoType_ECDSA, R: R, S: S}
log.Print(req)

// 使用gprc 远程调用接口调用远程 server 端register new user 方法
conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
	log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewECAAClient(conn)

token, err := c.RegisterUser(context.Background(), req)
if err != nil {
	return err
}

if token == nil {
	return errors.New("Failed to obtain token")
}

//need the token for later tests
user.enrollPwd = token.Tok

Then I get the error  :   panic:runtime error : invalid memory address or nil pointer dereference.By making breakpoints ,I find it  crashed at   hash :=primitives.NewHash()

Here is my error log.
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x476016]
goroutine 1 [running]:
panic(0x931180, 0xc82000a110)
/opt/go/src/runtime/panic.go:464 +0x3e6
github.com/hyperledger/fabric/core/crypto/primitives.NewHash(0x0, 0x0)
/opt/gopath/src/github.com/hyperledger/fabric/core/crypto/primitives/hash.go:41 +0x26
main.registerUser(0x9dc8a0, 0x5, 0xc90610, 0xc, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/opt/gopath/src/github.com/hyperledger/fabric/membersrvc/ca_client/main.go:169 +0x357
main.main()
/opt/gopath/src/github.com/hyperledger/fabric/membersrvc/ca_client/main.go:209 +0x40
exit status 2

@CocoaWang can you describe how reproduce the error? Sounds as if you haven't initialized the default hash.  Use this to initialize the security level before invoke hash := primitives.NewHash() in your code. For example to use SHA3 and length 256 bits use primitives.SetSecurityLevel("SHA3", 256). I understand that you are developing a custom client, you can see how this is initialized in regular client here.

@andresgaragiola  Hi!Thanks you first.I tried  hash:=primitives.SetSecurityLevel("SHA3",256) ,but get  the  same error .
It's true that  I am developing a custom client to register a new user which is not in the membersrvc.yaml
#Reproduce
I follow the  registerUser method in  eca_test.go file.And I changes
token, err := ecaa.RegisterUser(context.Background(), req)
to
conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewECAAClient(conn)
token, err := c.RegisterUser(context.Background(), req)

So I can use grpc service to register a new user.

@andresgaragiola  Hi ,thanks for you answer.You are right. My problem is that the Initialization is not sufficient.
I am working on registering a new user.And I found I can make a grpc client.Then I follow the eca_test.go 's  registerUser method separately in a new go file.Then I get the panic error.How should I make the initialization properly?

@andresgaragiola  Thanks again! I have found my error. I add some initialization in my code.Now I get some other errors.When I send my request to member service server / ecaa.go/RegisterUser method  like this :
token, err := c.RegisterUser(context.Background(), req)
I get: rpc error : code=2 desc = "sql: no rows in result set "

@CocoaWang can you send us the variable "req" attributes? Does this error occurs in the first RegisterUser invocation?

@andresgaragiola  Here is my code that I generate  my req :
////registerUser(testAdmin, &testUser)
hash := primitives.NewHash()
raw, _ := proto.Marshal(req)
hash.Write(raw)
//初始化 ecdsa
curve := primitives.GetDefaultCurve()
fmt.Printf("Default Curve %v \n", curve)
signPriv, err := primitives.NewECDSAKey()
testAdmin.enrollPrivKey = signPriv
r, s, err := ecdsa.Sign(rand.Reader, testAdmin.enrollPrivKey, hash.Sum(nil))
if err != nil {
	msg := "Failed to register user. Error (ECDSA) signing request: " + err.Error()
	log.Print(msg)
	//return errors.New(msg)
}
R, _ := r.MarshalText()
S, _ := s.MarshalText()
req.Sig = &pb.Signature{Type: pb.CryptoType_ECDSA, R: R, S: S}
log.Print(req)

The req result is
id:<id:"testUser" > role:CLIENT account:"institution_a" affiliation:"00001" registrar:<id:<id:"admin" > > sig:<r:"65574467999467688793422813304123357906562740277044923147984373162931017635525" s:"34880349115322446274243610531576160277660463394208669959616610631163344889105" >
Then I run  :
conn, err := grpc.Dial(address, grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := pb.NewECAAClient(conn)
token, err := c.RegisterUser(context.Background(), req)
Then I get the error log.

@andresgaragiola   By checking the source code,I find that the error occurs  in here
raw, err := ecaa.eca.readCertificateByKeyUsage(registrar, x509.KeyUsageDigitalSignature)
This queries the ca database to inquire registar ,but find nothing which led to a error. The registar I use  is   User{enrollID: "admin", enrollPwd: []byte("Xurw3yU9zI0l")}

@andresgaragiola  Hi! I just solved this error.It's because that I did not enroll my registar before registering my new user.
Thanks for your help!I just finish registering a new user  by using the gRPC method.

@CocoaWang, @andresgaragiola: great. Can we close this issue (if all is well...) ?

@JonathanLevi  Sure.Eeverything is ok.


Fix Objective C support (Swift 3 inference in Swift 4)

filename.m:910:47: Swift initializer 'ImageSource.init(image:)' uses '@objc' inference deprecated in Swift 4; add '@objc' to provide an Objective-C entrypoint

Since there are no explicit @objc attributes and Swift 4 compiler no longer agrees to silently infer them automatically, the library no longer compatible with Objective C apps. You either get the "inference deprecated" error, if you force the Swift 3 inference, or the "unknown selector", e.g.
filename.m:910:47: No visible @interface for 'ImageSource' declares the selector 'initWithImage:'

...if you don't.

Same issue.. :(

I added @objcMembers annotations where possible so it should now generate those Objective-C interfaces. Check out the version 1.4.1.


fix: render transformed element position bug

Feature

[x] Render transformed element position bug

Bug demo
View link https://jsfiddle.net/yugasun/pxy1ysbL, and click capture button.

@yugasun hello! Thank you for your PR. I need it too for my project.
@niklasvh please aprove it

You can use the onclone option to remove any styles, such as transforms. Example: https://github.com/niklasvh/html2canvas/issues/1448


[IN PROGRESS] Update cuML API

starting to locate and update hardcoded RAPIDS versions and outdated usages (i.e., train_test_split)

not including any databricks files

@sarahyurick thanks! Is this ready for review?

yes, the only examples I didn't update were:

mlflow/local_environment/notebooks/databricks_mlflow_rapids.ipynb
databricks/notebooks/rapids_airline_hyperopt.ipynb
Since they are related to Databricks.

also for reference, I generated a list of all files with the RAPIDS version hardcoded (e.g. if a file contains the string '21.06', 'rapidsai:0.19', etc.).
hardcoded_rapids.txt
I haven't updated any of the version numbers, though

Great we can leave the other two notebooks since those are based on previous DB setup, and that file is useful :)

ok, should I start updating any of the RAPIDS versions or just leave as is for now?

Can you open another PR to update the RAPIDS versions, this one should be good to go


Comparison option "Show key-values with different content only" doesn't work

In Azure Portal, in App Configuration ressource comparison of keys, the option "Show key-values with different content only" doesn't work. All data are displayed.
So, the comparison of key-values between to label is impossible !

Just want to clarify, are you trying to compare key-values with same values but different labels? If so, then this is expected because the option "Show key-values with different content only" will only filter out those with exact same key, value, label and content type.

Yes, I want to compare values of the same keys in two labels differents.
It is the goal, to compare values between two environment (two labels) and check if values are correct.
If we can't that, what is the interest of this comparison ?

The compare blade is designed to show users all differences between key-values include labels. We also add an info icon in front of all differences and hopefully it can help you catch the different values easier.

The option "Show key-values with different content only", as its name indicates, is only for keys and values, not for labels.
If we compare the keys and values between 2 different labels (known in advance) with the option "Show key-values with different content only", we expect to see only lines with different keys or different values. If you also compare the labels, then the option doesn't work any more because you end up with all the lines (with all key-values without filter).
If we want all the key-values, then we just have to uncheck the option.

@Philippe85 the scenario you described makes sense to me. If I understand you correctly, you expect, in my example below, mykey does not show up in the comparison list because it has the same value for the two labels (dev and test) I am comparing.

Yes. It is exact. Not show keys with same values. Show only keys with differences.

The options "Hide matching key-value pairs" and "Hide all matching keys" are incomprehensible.
What is the meaning of "matching" here !

"Matching" here is to describe the two compared elements having the same fields (so both the same key and value field for "Hide matching key-value pairs", or only the same key in "Hide all matching keys"). "Matching" does not consider differences in labels.
So to compare to the example above that Zhenlan has, "Hide matching key-value pairs" will remove _mykey_from view, while "Hide all matching keys" will remove both mykey and mykey2.

We already have the option "Hide matching key-value pairs" to achieve it. I think the issue left now is the compare options are not easy to understand.

The options "Hide matching key-value pairs" and "Hide all matching keys" are incomprehensible.

I'll follow up with the options naming.


Best Time to Buy and Sell Stock II

Description of the Problem
Say you have an array prices for which the ith element is the price of a given stock on day i.
Design an algorithm to find the maximum profit. You may complete as many transactions as you like (i.e., buy one and sell one share of the stock multiple times).
Note: You may not engage in multiple transactions at the same time (i.e., you must sell the stock before you buy again).
Example 1:
Input: [7,1,5,3,6,4]
Output: 7
Explanation: Buy on day 2 (price = 1) and sell on day 3 (price = 5), profit = 5-1 = 4.
Then buy on day 4 (price = 3) and sell on day 5 (price = 6), profit = 6-3 = 3.
Code
    def maxProfit(self, prices: List[int]) -> int:

        s =0
        for i in range(1,len(prices)):
            if prices[i]> prices[i-1]:
                s +=prices[i]-prices[i-1]
        return s

Link To The LeetCode Problem
https://leetcode.com/problems/best-time-to-buy-and-sell-stock-ii/
I would like to contribute well commented and explained code for this problem. Thanks

Please assign this issue to me

@tejasbirsingh Waiting for your PR 😸


Update Rotate-DkimSigningConfig.md

https://github.com/MicrosoftDocs/office-docs-powershell/issues/4364

Docs Build status updates of commit b98bf47:
:clock10: Pending: waiting for processors (9 builds ahead of you)
:warning: Docs Build is busy, currently there are 9 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit b98bf47:
:clock10: Full build: the average full build time is 11 min(s) 47 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit b98bf47:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/antispam-antimalware/Rotate-DkimSigningConfig.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 741c6f0:
:clock10: Full build: the average full build time is 11 min(s) 47 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 741c6f0:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/antispam-antimalware/Rotate-DkimSigningConfig.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 5b60cf6:
:clock10: Full build: the average full build time is 11 min(s) 47 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 5b60cf6:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/antispam-antimalware/Rotate-DkimSigningConfig.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


fly pause-resource / unpause-resource

Would it be possible to add a fly pause|unpause-resource command?
It seems it is currently only possible from the web-ui to pause input-resources for jobs/pipelines, but not through fly.

yep this would be pretty straightforward.
On Fri, May 13, 2016, 8:26 AM Concourse Bot<EMAIL_ADDRESS>wrote:

Hi there!
We use Pivotal Tracker to provide visibility into what our team is working
on. A story for this issue has been automatically created.
The current status is as follows:

#119537761 https://www.pivotaltracker.com/story/show/119537761 fly
pause-resource / unpause-resource

This comment, as well as the labels on the issue, will be automatically
updated as the status in Tracker changes.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/concourse/fly/issues/85#issuecomment-219076296


Revert "Test the GitHub Actions workflows"

Reverts ZEGUK/github-actions-continuous-delivery-azure#3

close


IsValidDate Not hire when dateFormat is set

Hi,
The function IsValidDate is not hire when we set a dateFormat
example : http://codepen.io/anon/pen/dpyEGr?editors=1112
In the example, we want limit month selector, without this function we can't...

Hi @katzuta,
I agree, you should be able to use a dateFormat and isValidDate at the same time. I'll see what I can find 🔬. Feel free to look at it yourself as well, if you'd like.

Hey, it turns out there was a PR fixing this that was not merged. I merged it and bumped the version, so when the creator of this package (who has access to the npm repo) publishes it to npm then you can download it it. It shouldn't take more than a day or two for him to do that.
I'll close this for now and if you're still having problems with 2.7.0 feel free to open it.


ELB auditor can't parse ICMP ingress rule

$ monkey find_changes -a acct
[...]
Traceback (most recent call last):
  File "/usr/local/src/security_monkey/venv/bin/monkey", line 11, in <module>
    load_entry_point('security-monkey', 'console_scripts', 'monkey')()
  File "/usr/local/src/security_monkey/security_monkey/manage.py", line 695, in main
    manager.run()
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/Flask_Script-0.6.3-py2.7.egg/flask_script/__init__.py", line 397, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/Flask_Script-0.6.3-py2.7.egg/flask_script/__init__.py", line 376, in handle
    return handle(app, *positional_args, **kwargs)
  File "/usr/local/src/security_monkey/venv/lib/python2.7/site-packages/Flask_Script-0.6.3-py2.7.egg/flask_script/commands.py", line 145, in handle
    return self.run(*args, **kwargs)
  File "/usr/local/src/security_monkey/security_monkey/manage.py", line 80, in find_changes
    sm_find_changes(account_names, monitor_names)
  File "/usr/local/src/security_monkey/security_monkey/scheduler.py", line 57, in find_changes
    audit_changes(accounts, monitor_names, False, debug)
  File "/usr/local/src/security_monkey/security_monkey/scheduler.py", line 103, in audit_changes
    _audit_changes(account, monitor.auditors, send_report, debug)
  File "/usr/local/src/security_monkey/security_monkey/scheduler.py", line 135, in _audit_changes
    au.audit_objects()
  File "/usr/local/src/security_monkey/security_monkey/auditor.py", line 670, in audit_objects
    method(item)
  File "/usr/local/src/security_monkey/security_monkey/auditors/elbv2.py", line 72, in check_internet_facing
    if self._issue_matches_listeners(alb, issue):
  File "/usr/local/src/security_monkey/security_monkey/auditor.py", line 262, in _issue_matches_listeners
    from_port = to_port = int(port)
ValueError: invalid literal for int() with base 10: '0--1'

Print debugging showed me what this was crashing on.
issue.notes -- Entity: [cidr:<IP_ADDRESS>/0] Access: [ingress:icmp:0--1]

The attached SG seems to be:

Looks like we need to check for icmp, and handle differently.

Looking into this.

Filed #894 to fix this.

I've merged #894.  Issue should be fixed now.

Confirmed working. Thanks for the fix!


chore(main): release 1.2.0

:robot: I have created a release beep boop
1.2.0 (2024-01-28)
Features

update GitHub Actions dev container workflow (#22) (462aa82)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/jwbennet/project-base/releases/tag/v1.2.0 :sunflower:


Toolbar overflow

show toolbar in a better way when not enough room for items, in this way i only show the items that fit the available space, and what remains i show it once the user click on the More button (3 dots icon) at the bottom of the toolbar

@hamed-musallam thanks for this PR. We are exploring the possibility to simply deal with the overflow using CSS in order to keep the code simple and keep this PR in stand-by for the moment.

Fixed in https://github.com/cheminfo/analysis-ui-components/commit/6c038ae97dafa868c7b00a63c9640e93d00f071f
https://cheminfo.github.io/analysis-ui-components/?path=/story/layout-toolbar--vertical-toolbar


cast_assoc(..., required: true) should have more options

It should both allow checking the existence of a field and if there are at least one item in the list.

The plan is for required: true to simply check for non-nil and validate_length to check for a number of entries. Btw @ericmj, can't you use validate_required and validate_length today to get the behaviour that you want?

You will need to call the validation functions AFTER you cast. Make sure to
not pass :required to cast and let me know how it goes. :)
--
José Valim
www.plataformatec.com.br
Skype: jv.ptec
Founder and Director of R&D

Thanks, I will try that out.

release_changeset
|> cast_assoc(:requirements)
|> validate_required(:requirements)

The validate_required/2 above does not seem to have any effect in my tests.

Oh... it does not work because it is always an empty list in the struct (or the NotLoaded struct). Now I remember why we need required: true in cast_assoc!

To solve this, we should raise if validate_required receives a NotLoaded struct.


Idea - Use OpenAI functions fine tunings for param parsing

Love the concept, but ran into issues with param parsing on dynamic routes with openAI, even with simple functions, when compared to running a completion on their functions fine tuning. There's likely an optimal way to parse function params using openAI's fine tuning and format, rather than having to prompt engineer 3.5-turbo into a JSON function call.
Obviously it's a bit less agnostic to foundational model, but could be dealt with via inheritance, and would make the "easy mode" work a lot better out of the box :)

Hi @hardcorebadger , thank you for raising this. We do something similar when using local models, by constraining output using GBNF.
For functions fine-tuning, is this example reflective of what you're referring to, where function signature is being passed to OpenAI as a Tool?
# Example dummy function hard coded to return the same weather
# In production, this could be your backend API or an external API
def get_current_weather(location, unit="fahrenheit"):
    """Get the current weather in a given location"""
    if "tokyo" in location.lower():
        return json.dumps({"location": "Tokyo", "temperature": "10", "unit": unit})
    elif "san francisco" in location.lower():
        return json.dumps({"location": "San Francisco", "temperature": "72", "unit": unit})
    elif "paris" in location.lower():
        return json.dumps({"location": "Paris", "temperature": "22", "unit": unit})
    else:
        return json.dumps({"location": location, "temperature": "unknown"})

def run_conversation():
    # Step 1: send the conversation and available functions to the model
    messages = [{"role": "user", "content": "What's the weather like in San Francisco, Tokyo, and Paris?"}]
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA",
                        },
                        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                    },
                    "required": ["location"],
                },
            },
        }
    ]
    response = client.chat.completions.create(
        model="gpt-3.5-turbo-0125",
        messages=messages,
        tools=tools,
        tool_choice="auto",  # auto is default, but we'll be explicit
    )

Looking into implementing this:

Hi @hardcorebadger , thank you for raising this. We do something similar when using local models, by constraining output using GBNF.
For functions fine-tuning, is this example reflective of what you're referring to, where function signature is being passed to OpenAI as a Tool?
# Example dummy function hard coded to return the same weather
# In production, this could be your backend API or an external API
def get_current_weather(location, unit="fahrenheit"):
    """Get the current weather in a given location"""
    if "tokyo" in location.lower():
        return json.dumps({"location": "Tokyo", "temperature": "10", "unit": unit})
    elif "san francisco" in location.lower():
        return json.dumps({"location": "San Francisco", "temperature": "72", "unit": unit})
    elif "paris" in location.lower():
        return json.dumps({"location": "Paris", "temperature": "22", "unit": unit})
    else:
        return json.dumps({"location": location, "temperature": "unknown"})

def run_conversation():
    # Step 1: send the conversation and available functions to the model
    messages = [{"role": "user", "content": "What's the weather like in San Francisco, Tokyo, and Paris?"}]
    tools = [
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g. San Francisco, CA",
                        },
                        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                    },
                    "required": ["location"],
                },
            },
        }
    ]
    response = client.chat.completions.create(
        model="gpt-3.5-turbo-0125",
        messages=messages,
        tools=tools,
        tool_choice="auto",  # auto is default, but we'll be explicit
    )

PR for this here: https://github.com/aurelio-labs/semantic-router/pull/258

Exciting stuff, will try it out!!


Add lifecycle state checks

On top of #124, diffed against master. This PR can contain all additional checks we add on lifecycle states or whether we should have them at all.
The only real diff ATM is reverting the removal of checks

Codecov Report

Merging #127 (15198ab) into master (38f793d) will decrease coverage by 0.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #127      +/-   ##
==========================================
- Coverage   37.63%   37.56%   -0.08%
==========================================
  Files          30       15      -15
  Lines        3762     1874    -1888
  Branches     2354     1176    -1178
==========================================
- Hits         1416      704     -712
+ Misses        246      121     -125
+ Partials     2100     1049    -1051

Flag
Coverage Δ

unittests
37.56% <ø> (-0.08%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...ller/test/test_load_forward_command_controller.cpp

...nt_state_controller/src/joint_state_controller.cpp

...ontroller/test/test_forward_command_controller.cpp

...int_trajectory_controller/test/test_trajectory.cpp

...ory_controller/src/joint_trajectory_controller.cpp

...ntroller/test/test_load_joint_state_controller.cpp

...ory_controller/src/joint_trajectory_controller.cpp

...ory_controller/test/test_trajectory_controller.cpp

...mand_controller/src/forward_command_controller.cpp

...ectory_controller/test/test_trajectory_actions.cpp

... and 35 more

Since the lifecycle changes are enforced by the controller manager, I don't think we need to check whether we're being used properly.
If we do need, we need to add on all controllers, and therefore will be copied by all future controller developers.

I wouldn't want to spend time on this given the understanding that we want to go back to lifecycle nodes at some point.

I've adjusted tests in #124 to make it pass without this


Import PJ SAS images into images database table, update and remove ol…

…d SQL dumps

 Root Card: Refactor the Backend Code

 (2) Import old product image data in to the new images table


clickhouse suddenly shut down by "bad_alloc" error

hello
I execute a “group by” sql to seach data from a replicatedMergeeTree table which has 200000000 rows，and after a while the clickhouse client responses "Status:-1" to me，then I cann't execute any more sql.
then I found that clickhouse progress shut down for a while and restart by itself after serveral minutes later.
I found the error log below:
clickhouse error log:
----------------------------------split line---------------------------------------------------------
Code: 1001, type: std::bad_alloc, e.what() = std::bad_alloc (from xxx.xxx.xxx.xxx) (in query:  /TABIX_QUERY_ID_PLo6aU3y/ select
2017.11.13 13:18:13.131854 [ 6999 ]  HTTPHandler: std::exception. Code: 1001, type: std::bad_alloc, e.what() = std::bad_alloc
2017.11.13 13:18:14.313290 [ 7065 ]  BaseDaemon: ########################################
2017.11.13 13:18:14.313323 [ 7065 ]  BaseDaemon: (from thread 89) Received signal Segmentation fault (11).
2017.11.13 13:18:14.313343 [ 7065 ]  BaseDaemon: Address: 0x35da135000
2017.11.13 13:18:14.324545 [ 7065 ]  BaseDaemon: 0. clickhouse-server(tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int)+0xe3) [0x30151c3]
2017.11.13 13:18:14.324574 [ 7065 ]  BaseDaemon: 1. clickhouse-server(tcmalloc::ThreadCache::Scavenge()+0x40) [0x3015320]
2017.11.13 13:18:14.324590 [ 7065 ]  BaseDaemon: 2. clickhouse-server(tc_delete_nothrow+0x386) [0x35de486]
2017.11.13 13:18:14.324623 [ 7065 ]  BaseDaemon: 3. clickhouse-server(void std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > >::_M_emplace_back_aux<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&>(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&)+0x15a) [0x11aecea]
2017.11.13 13:18:14.324651 [ 7065 ]  BaseDaemon: 4. clickhouse-server(DB::MergeTreeDataMerger::mergePartsToTemporaryPart(std::vector<std::shared_ptr<DB::MergeTreeDataPart const>, std::allocator<std::shared_ptr<DB::MergeTreeDataPart const> > >&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, DB::MergeListEntry&, unsigned long, long, DB::DiskSpaceMonitor::Reservation*, bool)+0x171b) [0x2c4be6b]
2017.11.13 13:18:14.324680 [ 7065 ]  BaseDaemon: 5. clickhouse-server(DB::StorageReplicatedMergeTree::executeLogEntry(DB::ReplicatedMergeTreeLogEntry const&)+0xf94) [0x2bd8434]
2017.11.13 13:18:14.324693 [ 7065 ]  BaseDaemon: 6. clickhouse-server() [0x2bdae2e]
2017.11.13 13:18:14.324715 [ 7065 ]  BaseDaemon: 7. clickhouse-server(DB::ReplicatedMergeTreeQueue::processEntry(std::function<std::shared_ptrzkutil::ZooKeeper ()>, std::shared_ptrDB::ReplicatedMergeTreeLogEntry&, std::function<bool (std::shared_ptrDB::ReplicatedMergeTreeLogEntry&)>)+0x3b) [0x2cbec1b]
2017.11.13 13:18:14.324730 [ 7065 ]  BaseDaemon: 8. clickhouse-server(DB::StorageReplicatedMergeTree::queueTask()+0x132) [0x2bc5ae2]
2017.11.13 13:18:14.324745 [ 7065 ]  BaseDaemon: 9. clickhouse-server(DB::BackgroundProcessingPool::threadFunction()+0x3ce) [0x2c22a1e]
2017.11.13 13:18:14.324764 [ 7065 ]  BaseDaemon: 10. clickhouse-server() [0x358b1ef]
2017.11.13 13:18:14.324777 [ 7065 ]  BaseDaemon: 11. /lib64/libpthread.so.0() [0x3af0c07aa1]
----------------------------------split line---------------------------------------------------------
Does this kind of error can cause clickhouse shuting down?and how can I to prevent this error happenning again?

How much memory do you allocated to CK?

I set the <max_memory_usage>80000000000</max_memory_usage> in  'users.xml'

@shernupup is this issue still relevant? how much physical memory your server had? were you actually running queries that require much memory to execute?


Address unit test flakiness

ETL-5122: HadoopUtilsTest.testSafeRenameRecursively: currently, the test waits for the shutdown of executorService without actually shutting it down.
ETL-5123: Increase timeouts in TestTunnelWithArbitraryTCPTraffic

@htran1 can you have a look?

Self-approving since all tests pass.


[twitch] Embedding subs (chat) failing

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2020.05.08
[x] I've checked that all provided URLs are alive and playable in a browser
A bit of background: Twitch has "Video on demand"=VOD, basically whole archived streams, and "clips", 30-second segments. VOD has chat, I think clips don't. Since VOD disappears after a month, I archive it--but also you may need to find a new sample URL to test on depending on how long it takes to get to this, please don't just close the bug because the URL is expired.
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones
I did survey existing bugs. This may be related to #18593, I can't tell if that issue was fixed and just not cleaned up, but this is a new error message so I'm filing it as a new bug. I didn't see any other bugs that have this error message and the word "twitch", open or closed.

Verbose log
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://twitch.tv/videos/627004856', u'--no-playlist', u'--add-metadata', u'--all-subs', u'--embed-subs', u'--download-archive', u'ARCHIVE', u'-o', u'videos/twitch_%(id)s.%(ext)s']
[debug] Encodings: locale UTF-8, fs UTF-8, out None, pref UTF-8
[debug] youtube-dl version 2020.05.08
[debug] Python version 2.7.16 (CPython) - Linux-4.19.0-8-amd64-x86_64-with-debian-10.3
[debug] exe versions: ffmpeg 4.1.4-1, ffprobe 4.1.4-1
[debug] Proxy map: {}
[twitch:vod] 627004856: Downloading vod info JSON
[twitch:vod] 627004856: Downloading vod access token
[twitch:vod] 627004856: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[info] Writing video subtitles to: videos/twitch_v627004856.rechat.json
[debug] Invoking downloader on u'https://dqrpb9wgowsf5.cloudfront.net/9077bff1bbd844a1ea14_skizzleman_38182849504_1455302010/chunked/index-dvr.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 911
[download] Destination: videos/twitch_v627004856.mp4
[download] 100% of 3.58GiB in 06:22.12MiB/s ETA 00:00042:04
[debug] ffmpeg command line: ffprobe -show_streams 'file:videos/twitch_v627004856.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "videos/twitch_v627004856.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:videos/twitch_v627004856.mp4' -c copy -f mp4 '-bsf:a' aac_adtstoasc 'file:videos/twitch_v627004856.temp.mp4'
[ffmpeg] Adding metadata to 'videos/twitch_v627004856.mp4'
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:videos/twitch_v627004856.mp4' -c copy -metadata 'date=20200520' -metadata 'purl=https://twitch.tv/videos/627004856' -metadata 'artist=Skizzleman' -metadata 'title=Solo Skizz Sitcom' 'file:videos/twitch_v627004856.temp.mp4'
[ffmpeg] Embedding subtitles in 'videos/twitch_v627004856.mp4'
[debug] ffmpeg command line: ffmpeg -y -loglevel 'repeat+info' -i 'file:videos/twitch_v627004856.mp4' -i 'file:videos/twitch_v627004856.rechat.json' -map 0 -c copy -map '-0:s' -map '-0:d' '-c:s' mov_text -map '1:0' '-metadata:s:s:0' 'language=rechat' 'file:videos/twitch_v627004856.temp.mp4'
ERROR: file:videos/twitch_v627004856.rechat.json: Invalid data found when processing input
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2065, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 426, in run
    self.run_ffmpeg_multiple_files(input_files, temp_filename, opts)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 235, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
FFmpegPostProcessorError: file:videos/twitch_v627004856.rechat.json: Invalid data found when processing input

Description
As best I can tell, the flags --all-subs --embed-subs causes videos with chat to fail download on twitch.
Edit: Oh also, I believe the expected behavior on twitch is that "subs" should some version of twitch chat, embedded as a subtitle track, which isn't super obvious :)
Thanks everyone for maintaining this great program.

Because Twitch subs are in a format that can't be embedded.

Oh, okay. I thought this was working at some point, must be mistaken.
Is there any other way to shove them in the same file, like is done with youtube metadata?
Anyway feel free to close, although I do think it would be a plus if there was a better error message for unsupported operations like this.

Oh, okay. I thought this was working at some point, must be mistaken.
Is there any other way to shove them in the same file, like is done with youtube metadata?
Anyway feel free to close, although I do think it would be a plus if there was a better error message for unsupported operations like this.

You can close the issue since you opened it. Just click the button.
To answer your first question, you'd have to manually transcode the JSON subs format to SRT, then embed it into the file, but that won't look pretty like you think it will and I don't even think you can transcode to that format.

On 2020-06-06 20:26, Paul Henning wrote:

Anyway feel free to close, although I do think it would be a plus if
there was a better error message for unsupported operations like
this.

You can close the issue since you opened it. Just click the button.

I don't care to close this because I think there's still an action item
(print a better error message).
One contributor should please take charge and decide whether that should
be done (to avoid lack of responsibility), and close it as wontfix if
not :).

Oh, okay. I thought this was working at some point, must be
mistaken.
Is there any other way to shove them in the same file, like is done
with youtube metadata?

To answer your first question, you'd have to manually transcode the
JSON subs format to SRT, then embed it into the file, but that won't
look pretty like you think it will and I don't even think you can
transcode to that format.

By "shove them in the same file", I meant that for youtube,
--add-metadata adds youtube's JSON metadata into the .mkv somehow after
download (I assume as a metadata track, although it looks like there
might be an xattr option? I really need to learn more about how this
works). Sorry for being a bit vague the first time around. It seems like
that would be useful here, even if actual subtitle embedding isn't
possible. However this is a poorly-thought-out idea, and definitely a
separate issue, I was just thinking aloud.

JSON metadata is different from subtitles, they don't even describe the same things.

Yes. I was suggesting to use the same mechanism to include the 'XXXrechat.json' as a track as is used to include the youtube metadata JSON as a track. And I'm retracting my suggestion, in any case.
Feel free to add a check for the incompatible: twitch, --all-subs, and --embed-subs with a nice error message. I feel like this conversation isn't going anywhere productive so I'm closing this.


Multiple databases documentation

The CAIRIS web app supports the ability to create and open multiple databases, but this needs to be documented.

Superseded by Issue #113


now tests are runnable from any directory

fixes #49

@googlebot I signed it!

Tests are passing from any directory.  Thanks so much for your fix!


how can we check web version is changed and DO NOT download?

hi
I want to check the web version , and download them after user confirmed,
but chcp.fetchUpdate download all the files automatic.
can you make a method like chcp.checkUpdate which only download chcp.json and chco.manifest?
then I can know we have some files need tobe download?
thank you~~

Hi,
I think someone already asked for similar api method :) The problem with this approach is that Apple can reject your app. According to their policy you can't ask user if he want to update the app or not and perform the update with methods other then App Store.
Basically, Apple afraid that  user might get confused (bad UX) about what type of update is in there. That user should be always redirect to the App Store if he sees a message like "do you want to update?".
That's why this method is not implemented: you should just update the content and that's it. In that case you are fine.

As a workaround for now, if you want, you can use chcp.getVersionInfo(callback) method to get information about currently installed version. And then from your js code you can make a direct request to the chcp.json file on the server side to check, if new version is available. And then prompt user for update. But as I said in the last post - asking user can make Apple to reject your app.

thank you~~
that's ok,
I just don't want to update will user in a bad network. I'll try to check the network status and update through wifi for example.
I will not ask user for update, thank you for your answer, that's very useful for me!!
thanks a lot!

Your welcome :)


[drift_dev]: schema_steps does not work for Postgres databases

I have the following table users defined like:
class Users extends Table with CreatedAt {
  UuidColumn get id => customType(PgTypes.uuid).withDefault(genRandomUuid())();

  TextColumn get name => text()();

  TextColumn get email => text().unique()();

  TextColumn get password => text()();

  @override
  Set<Column> get primaryKey => {id};
}

Notice I am using getRandomUuid() as the default. The CreatedAt mixin has the following:
mixin CreatedAt on Table {
  TimestampColumn get createdAt =>
      customType(PgTypes.timestampWithTimezone).withDefault(now())();
}

With this column, I am using now() as the default.
Running dart run drift_dev schema steps after doing a dump generates the versions file, without the import for drift_postgres. Adding the import manually (which is a workaround I am willing to automate), the values of now() and getRandomUuid() do not adhere to the DriftAny type in the generated code. Here is a snippet of the generated code for a users table from the previous schema version which fails:
i1.GeneratedColumn<i1.DriftAny> _column_0(String aliasedName) =>
    i1.GeneratedColumn<i1.DriftAny>('created_at', aliasedName, false,
        type: i1.DriftSqlType.any, defaultValue: now()); // <- Error here
i1.GeneratedColumn<i1.DriftAny> _column_1(String aliasedName) =>
    i1.GeneratedColumn<i1.DriftAny>('id', aliasedName, false,
        type: i1.DriftSqlType.any, defaultValue: genRandomUuid()); // <- Error here
i1.GeneratedColumn<String> _column_2(String aliasedName) =>
    i1.GeneratedColumn<String>('name', aliasedName, false,
        type: i1.DriftSqlType.string);
i1.GeneratedColumn<String> _column_3(String aliasedName) =>
    i1.GeneratedColumn<String>('email', aliasedName, false,
        type: i1.DriftSqlType.string);
i1.GeneratedColumn<String> _column_4(String aliasedName) =>
    i1.GeneratedColumn<String>('password', aliasedName, false,
        type: i1.DriftSqlType.string);

Let me know if there's any more information I can provide
Environment information:
drift: 2.20.3
drift_dev: 2.20.3
drift_postgres: 1.3.0
postgres: 3.4.0

build.yaml
targets:
  drift:
    auto_apply_builders: false
    builders:
      drift_dev:analyzer:
        enabled: true
        options: &options
          store_date_time_values_as_text: true
          named_parameters: true
          sql:
            dialect: postgres
      drift_dev:modular:
        enabled: true
        options: *options

  $default:
    dependencies:
      - ":drift"
    builders:
      drift_dev:
        enabled: false

Thanks for the report! These issues sound fixable and I'll take a look, but you'll probably won't get far with step-by-step migrations on postgres:

The schema verification code used in tests does not support postgres (neither as a dialect nor can we reliably obtain the actual schema at runtime).
Step-by-step migrations run a user_version pragma after each step, which will not work on postgres.
In fact, most drift migration APIs (outside of creating tables) won't work on postgres. Migrator.alterTable certainly won't, for instance.

We should still fix this for users who are using custom types to share a database between postgres and sqlite3, but the migration tooling for Postgres is just fairly incomplete at the moment. At the moment, I suggest extracting your drift schema into a separate file (drift dev schema export!) and then use a dedicated migration tool for Postgres.

Thanks for the information :) I appreciate it


Connection losses with many nodes/properties

Hi there,
first of all thanks for this library, I really like how easy it is to build homie devices!
When I tried to use it for my first real-world project, I ran into the following issue, though.
The device has 9 nodes with two properties each (9 motors with speed and direction property).
The problem is that the device tries to publish its nodes and its properties during startup but never accomplishes it since the connection is lost and the property publishing starts all over again.
I first thought it's a problem of my nodes, but I managed to reproduce the issue with the led example.
import settings

from machine import Pin

from homie.constants import FALSE, TRUE, BOOLEAN, INTEGER
from homie.device import HomieDevice
from homie.node import HomieNode
from homie.property import HomieProperty

class LED(HomieNode):

    # Reversed values for the esp8266 boards onboard led
    ONOFF = {FALSE: 1, TRUE: 0}

    def __init__(self, id=0, name="Onboard LED", pin=0):
        super().__init__(id="led%d" % id, name=name, type="LED")

        self.p_power = HomieProperty(
            id="power",
            name="LED Power",
            settable=True,
            datatype=BOOLEAN,
            default=FALSE,
            on_message=self.on_power_msg,
        )
        self.add_property(self.p_power)

        self.directionProperty = HomieProperty(
            id="direction",
            name="direction",
            settable=True,
            datatype=INTEGER,
            default=30,
            on_message=self.direction_msg,
        )
        self.add_property(self.directionProperty)

    def on_power_msg(self, topic, payload, retained):
        print("power_msg %s" % self.id)

    def direction_msg(self, topic, payload, retained):
        print("direction_msg %s" % self.id)

def main():
    # Homie device setup
    homie = HomieDevice(settings)

    # Add LED node to device
    homie.add_node(LED())
    homie.add_node(LED(1, "LED 1"))
    homie.add_node(LED(2, "LED 2"))
    homie.add_node(LED(3, "LED 3"))
    homie.add_node(LED(4, "LED 4"))
    homie.add_node(LED(5, "LED 5"))
    homie.add_node(LED(6, "LED 6"))
    homie.add_node(LED(7, "LED 7"))
    homie.add_node(LED(8, "LED 8"))
    homie.add_node(LED(9, "LED 9"))
    homie.add_node(LED(10, "LED 10"))
    homie.add_node(LED(11, "LED 11"))

    # run forever
    homie.run_forever()

if __name__ == "__main__":
    main()

This is what happens:
CONNECTION HANDLER!
MQTT SUBSCRIBE: devices/microhomie1/$mpy
MQTT PUBLISH: devices/microhomie1/$homie --> b'4.0.0'
MQTT SUBSCRIBE: devices/microhomie1/led0/power
MQTT SUBSCRIBE: devices/microhomie1/led0/direction
MQTT SUBSCRIBE: devices/microhomie1/led1/power
MQTT SUBSCRIBE: devices/microhomie1/led1/direction
MQTT SUBSCRIBE: devices/microhomie1/led2/power
MQTT SUBSCRIBE: devices/microhomie1/led2/direction
MQTT SUBSCRIBE: devices/microhomie1/led3/power
MQTT SUBSCRIBE: devices/microhomie1/led3/direction
MQTT SUBSCRIBE: devices/microhomie1/led4/power
MQTT SUBSCRIBE: devices/microhomie1/led4/direction
MQTT SUBSCRIBE: devices/microhomie1/led5/power
MQTT SUBSCRIBE: devices/microhomie1/led5/direction
MQTT SUBSCRIBE: devices/microhomie1/led6/power
MQTT SUBSCRIBE: devices/microhomie1/led6/direction
MQTT SUBSCRIBE: devices/microhomie1/led7/power
MQTT SUBSCRIBE: devices/microhomie1/led7/direction
MQTT SUBSCRIBE: devices/microhomie1/led8/power
MQTT SUBSCRIBE: devices/microhomie1/led8/direction
MQTT SUBSCRIBE: devices/microhomie1/led9/power
MQTT SUBSCRIBE: devices/microhomie1/led9/direction
MQTT SUBSCRIBE: devices/microhomie1/led10/power
MQTT SUBSCRIBE: devices/microhomie1/led10/direction
MQTT SUBSCRIBE: devices/microhomie1/led11/power
MQTT SUBSCRIBE: devices/microhomie1/led11/direction
MQTT PUBLISH: devices/microhomie1/$name --> b'microhomie1'
MQTT SUBSCRIBE: devices/microhomie1/led0/power/set
MQTT MESSAGE: devices/microhomie1/led0/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led0/power
MQTT SUBSCRIBE: devices/microhomie1/led0/direction/set
MQTT MESSAGE: devices/microhomie1/led0/direction --> 30, True
direction_msg led0
MQTT UNSUBSCRIBE: devices/microhomie1/led0/direction
MQTT SUBSCRIBE: devices/microhomie1/led1/power/set
MQTT MESSAGE: devices/microhomie1/led1/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led1/power
MQTT SUBSCRIBE: devices/microhomie1/led1/direction/set
MQTT MESSAGE: devices/microhomie1/led1/direction --> 30, True
direction_msg led1
MQTT UNSUBSCRIBE: devices/microhomie1/led1/direction
MQTT SUBSCRIBE: devices/microhomie1/led2/power/set
MQTT MESSAGE: devices/microhomie1/led2/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led2/power
MQTT SUBSCRIBE: devices/microhomie1/led2/direction/set
MQTT MESSAGE: devices/microhomie1/led2/direction --> 30, True
direction_msg led2
MQTT UNSUBSCRIBE: devices/microhomie1/led2/direction
MQTT SUBSCRIBE: devices/microhomie1/led3/power/set
MQTT MESSAGE: devices/microhomie1/led3/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led3/power
MQTT SUBSCRIBE: devices/microhomie1/led3/direction/set
MQTT MESSAGE: devices/microhomie1/led3/direction --> 30, True
direction_msg led3
MQTT UNSUBSCRIBE: devices/microhomie1/led3/direction
MQTT SUBSCRIBE: devices/microhomie1/led4/power/set
MQTT MESSAGE: devices/microhomie1/led4/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led4/power
MQTT SUBSCRIBE: devices/microhomie1/led4/direction/set
MQTT MESSAGE: devices/microhomie1/led4/direction --> 30, True
direction_msg led4
MQTT UNSUBSCRIBE: devices/microhomie1/led4/direction
MQTT SUBSCRIBE: devices/microhomie1/led5/power/set
MQTT MESSAGE: devices/microhomie1/led5/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led5/power
MQTT SUBSCRIBE: devices/microhomie1/led5/direction/set
MQTT MESSAGE: devices/microhomie1/led5/direction --> 30, True
direction_msg led5
MQTT UNSUBSCRIBE: devices/microhomie1/led5/direction
MQTT SUBSCRIBE: devices/microhomie1/led6/power/set
MQTT MESSAGE: devices/microhomie1/led6/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led6/power
MQTT SUBSCRIBE: devices/microhomie1/led6/direction/set
MQTT MESSAGE: devices/microhomie1/led6/direction --> 30, True
direction_msg led6
MQTT UNSUBSCRIBE: devices/microhomie1/led6/direction
MQTT SUBSCRIBE: devices/microhomie1/led7/power/set
MQTT MESSAGE: devices/microhomie1/led7/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led7/power
MQTT SUBSCRIBE: devices/microhomie1/led7/direction/set
MQTT MESSAGE: devices/microhomie1/led7/direction --> 30, True
direction_msg led7
MQTT UNSUBSCRIBE: devices/microhomie1/led7/direction
MQTT SUBSCRIBE: devices/microhomie1/led8/power/set
MQTT MESSAGE: devices/microhomie1/led8/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led8/power
MQTT SUBSCRIBE: devices/microhomie1/led8/direction/set
MQTT MESSAGE: devices/microhomie1/led8/direction --> 30, True
direction_msg led8
MQTT UNSUBSCRIBE: devices/microhomie1/led8/direction
MQTT SUBSCRIBE: devices/microhomie1/led9/power/set
MQTT MESSAGE: devices/microhomie1/led9/power --> false, True
MQTT UNSUBSCRIBE: devices/microhomie1/led9/power
CONNECTION HANDLER!

It then keeps looping...
I'm using version 3.0.2 and MicroPython version v1.14.
Thanks,
Axel

Hi Axel, do you run this on an esp8266, this could be to much for this microcontroller. But I need to try this by my own. You can try the ArrayNode from the feature/arraynode branch, if you want. I think the ArrayNode could be perfect for what you want to archiv. For example: https://github.com/microhomie/microhomie/blob/feature%2Farraynode/examples/array/main.py
As the ArrayNode is a PoC I would be happy if you have feedback for me.
Best regards,
Rafael

https://github.com/microhomie/microhomie/discussions/51

Hi Rafael,
thanks for the quick reply! I forgot to mention... I'm using an esp32 wrover dev kit.
There must be some kind of send buffer overflow or something. It works with less nodes and also with less properties.
Thanks, I'm going to check out your ArrayNode feature! I will let you know...
Axel

Hi Rafael,
after long hours of debugging I found that my issue is not a Microhomie issue. The problem seems to be the way I added my recurring code. I'm going to post a question about this in the forum.
Unfortunately the ArrayNode feature doesn't work for me since auto discovery is important for my properties.
I'm closing this issue.
Thanks,
Axel


Couldn't parse field state from table epoch_sync_time. Cannot convert PersistLiteral to Text

node: 1.27.0
db-sync: commit 38769647f126b8c68083aaafe3bee1751789a055 on release/10.0.x
network: mainnet, shelley-qa, (this is related to new table epoch_sync_time so it is rather network "independent" issue)
Both components built with nix, as in instruction. Logs attached at the end of ticket.
After db-sync was successfully built and started for the first time on release/10.0.x branch I stopped it and then stopped node. After starting node again (no errors):

[artur-li:cardano.node.ChainDB:Notice:56] [2021-05-31 16:29:57.66 UTC] Chain extended, new tip: 49cbe5b80894edeb86b6d69fe8098847de97c6e609b0731b2a06707b4f5e70dd at slot 30912306
[artur-li:cardano.node.ChainDB:Notice:56] [2021-05-31 16:30:21.78 UTC] Chain extended, new tip: 47621563a245febff8861ecbf5652f8cc30ee2d44fd6a3406dacb8c2d95de785 at slot 30912330

I started db-sync again and there are errors in the logs informing about parsing issues, this keeps repeating - subscription being restarted and then again parsing error showing up, in the loop:

db-sync-node:Info:54] [2021-05-31 16:30:28.50 UTC] thisIsAnUglyHack: Main thead
[db-sync-node:Error:54] [2021-05-31 16:30:28.50 UTC] runDBThread: PersistMarshalError "Couldn't parse field state from table epoch_sync_time. Cannot convert PersistLiteral to Text"
[db-sync-node:Error:49] [2021-05-31 16:30:28.50 UTC] ChainSyncWithBlocksPtcl: PersistMarshalError "Couldn't parse field state from table epoch_sync_time. Cannot convert PersistLiteral to Text"
[db-sync-node.Mux:Info:46] [2021-05-31 16:30:28.50 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: State: Dead
[db-sync-node.Mux:Notice:46] [2021-05-31 16:30:28.50 UTC] Bearer on ConnectionId {localAddress = LocalAddress "", remoteAddress = LocalAddress "state-node-mainnet/node.socket"} event: Miniprotocol MiniProtocolNum 5 InitiatorDir terminated with exception PersistMarshalError "Couldn't parse field state from table epoch_sync_time. Cannot convert PersistLiteral to Text"
[db-sync-node.Subscription:Error:45] [2021-05-31 16:30:28.50 UTC] Identity Application Exception: LocalAddress "../cardano-node/state-node-mainnet/node.socket" PersistMarshalError "Couldn't parse field state from table epoch_sync_time. Cannot convert PersistLiteral to Text"
[db-sync-node.ErrorPolicy:Error:6] [2021-05-31 16:30:28.50 UTC] IP LocalAddress "../cardano-node/state-node-mainnet/node.socket" ErrorPolicyUnhandledApplicationException (PersistMarshalError "Couldn't parse field state from table epoch_sync_time. Cannot convert PersistLiteral to Text")
[db-sync-node.Subscription:Notice:44] [2021-05-31 16:30:29.51 UTC] Identity Restarting Subscription after 1.767893481s desired valency 1 current valency 0
[db-sync-node.Subscription:Notice:44] [2021-05-31 16:30:37.75 UTC] Identity Starting Subscription Worker, valency 1

and on the node side those errors appeared right after starting db-sync:

[artur-li:cardano.node.ChainDB:Notice:56] [2021-05-31 16:30:21.78 UTC] Chain extended, new tip: 47621563a245febff8861ecbf5652f8cc30ee2d44fd6a3406dacb8c2d95de785 at slot 30912330
[artur-li:cardano.node.LocalErrorPolicy:Error:84] [2021-05-31 16:30:28.51 UTC] IP LocalAddress "" ErrorPolicyUnhandledApplicationException (MuxError (MuxIOException writev: resource vanished (Broken pipe)) "(sendAll errored)")
[artur-li:cardano.node.LocalErrorPolicy:Error:84] [2021-05-31 16:30:39.47 UTC] IP LocalAddress "" ErrorPolicyUnhandledApplicationException (MuxError MuxBearerClosed "<socket: 36> closed when reading data, waiting on next header True")

Stopping and starting services again did not help with that db-sync parsing error.
Logs:
db_sync_errors_mainnet_logs.txt
node_errors_mainnet_logs.txt

I was able to replicate this, but after that error is reported, db-sync seems to continue to work correctly.
Yes, I will fix this, but this error seems to be pretty benign.


Support a proper means for unsetting keys

Problem Description
In order to port a part of my justfile, I'm trying to bind tasks.clippy to cargo +nightly clippy with the environment variable CARGO_TARGET_DIR=target/nightly to keep it from clobbering the stable build artifact cache... but I ran into the following problems:

The default definition of the task tries to install the clippy crate and dies because my default compiler is stable.

I can't use the usual approach for unsetting an overlaid value from other languages because TOML has no provision for install_crate = null.

If I rename my task to tasks.clippyh as a test, I get a failed attempt to install cargo-+nightly

If I try setting script = ["cargo +nightly clippy"] in tasks.clippy, I get Invalid task, contains multiple actions, which is back to the same problem. (command = null not being valid TOML)

That last one does work if I name the task clippyh, but now I'm stuck with a conundrum. Do I...

Use a shell function which remaps cargo make clippy to cargo make clippy-real and passes through everything else, rather than simply alias cm=cargo make?
Use the nuclear option of skip_core_tasks = true and remove my primary reason for seeing cargo-make as more desirable than just. (Even with the predefined actions, you're going to be seeing several more issues filed by me about reasons cargo-make falls short of my needs.)

Code Example
[tasks.clippy]
# TODO: Some means of unsetting `command` so this works
script = ["cargo +nightly clippy"]
env = { "CARGO_TARGET_DIR" = "target/nightly" }

Error Stack/Info (if any)
[cargo-make] INFO - cargo-make 0.11.3
[cargo-make] INFO - Using Build File: Makefile.toml
[cargo-make] INFO - Task: clippy
[cargo-make] INFO - Setting Up Env.
[cargo-make] INFO - Running Task: init
[cargo-make] INFO - Running Task: clippy
[cargo-make] ERROR - Invalid task, contains multiple actions.

EDIT: I also just realized that removing the predefined install_crate will be important now that things like clippy are in the process of migrating from cargo-based distribution to rustup-based distribution.

if clippy are moving away from cargo to standalone executable, than I cannot automatically install it for you.
basically if its a cargo plugin, i look at the command and install it, if missing, so with clippy being not a cargo plugin, that would no longer happen.
so cargo-make tries to help you by auto installing what is missing, but ya, i see in this case that its hurting because clippy is kinda not related to the rust version you are running with.
Until than, you will want to disable that automatic installation that is done by cargo make for plugins, so i guess maybe a new attribute:
install = false
```
should resolve that and you can set it in your extended task in your makefile.toml. no?
that I can do pretty easily, but only if it really help you out.

by the way, did clippy already move away from cargo?
if so, maybe i'll just update the predefined makefile to use the new clippy without cargo.
that would mean, that nothing would be auto installed by cargo.
I'm also open for pull request if you have a good solution for the predefined clippy task that is in the internal makefile.

Until than, you will want to disable that automatic installation that is done by cargo make for plugins, so i guess maybe a new attribute:
install = false

should resolve that and you can set it in your extended task in your makefile.toml. no?
that I can do pretty easily, but only if it really help you out.

I'd prefer a more general solution, such as an inherit = false which allows me to redefine individual predefined commands from scratch. This may not be the only time where replacing a command with a script would be useful.

by the way, did clippy already move away from cargo?
if so, maybe i'll just update the predefined makefile to use the new clippy without cargo.

This internals thread says that there should now be a clippy-preview component in the nightly channel, but, no matter what I've tried, I can't get my rustup to acknowledge its existence.

inherit = false sounds like a cool solution and definitely useful!!!
I'll work on that. will probably be ready tomorrow or so...

I'm senile. that's already implemented via clear attribute

clear is now support on every task type, docs updated.
you can test it by using branch 0.12.0
please tell me if it works good for you.

closing as it is now officially released.


Added Code for Mux, Demux, Encoder, Decoder, Led, HexDisplay

Fixes #
Describe the changes you have made in this PR -
Dynamically generate Verilog Multiplexer, Demux, Encoder, Decoder.
Fixed NotGate so it works with multi-bit using Assign
Made LED and HexDisplay print with $display
Fixed ConstVal using assign
Implemented Counter and Random
Please use Tools->Convert Verilog
Screenshots of the changes (If any) -
Sample circuit

Sample generated Code

module Main(dmx_0, dmx_1, enc, enc_en, so, out_5, out_6, out_7, clk, mux_0, mux_1, si, inp_3, inp_4, inp_5, inp_6, inp_7);
  output dmx_0,  dmx_1,  enc,  enc_en,  out_5,  out_7;
  output [1:0] so, out_6;
  input clk, mux_0, mux_1, inp_3;
  input [1:0] si, inp_4, inp_5, inp_6;
  input [2:0] inp_7;
  wire clk, Decoder_0_out_0, Decoder_0_out_1, PriorityEncoder_0_out_0, PriorityEncoder_0_out_1, and_0_out, Multiplexer_0_out, Demultiplexer_0_out_0, Demultiplexer_0_out_1;
  wire [1:0] Splitter_1_inp, Multiplexer_1_out;
  wire [3:0] Counter_0_Value, Counter_0_Zero, const_0, Random_0_RandomValue;
  assign dmx_0 = Demultiplexer_0_out_0;
  assign dmx_1 = Demultiplexer_0_out_1;
  assign enc = PriorityEncoder_0_out_0;
  assign enc_en = PriorityEncoder_0_out_1;
  assign so = Splitter_1_inp;
  assign out_5 = and_0_out;
  assign out_6 = Multiplexer_1_out;
  assign out_7 = Counter_0_Zero;

  Counter #(4) Counter_0(Counter_0_Value, Counter_0_Zero, const_0, clk, Splitter_1_inp[0]);

  always @ (Counter_0_Value)
    $display("Counter_0_Value = %d", Counter_0_Value);

  Decoder2 Decoder_0(Decoder_0_out_0, Decoder_0_out_1, clk);
  PriorityEncoder2 PriorityEncoder_0(PriorityEncoder_0_out_0, PriorityEncoder_0_out_1, Decoder_0_out_0, Decoder_0_out_1);
  Multiplexer8 #(2) Multiplexer_1(Multiplexer_1_out, inp_4, inp_6, inp_4, inp_5, inp_5, inp_5, inp_6, inp_5, inp_7);
  and and_0(and_0_out, inp_3, inp_3);
  Multiplexer2 Multiplexer_0(Multiplexer_0_out, mux_0, mux_1, clk);
  Demultiplexer2 Demultiplexer_0(Demultiplexer_0_out_0, Demultiplexer_0_out_1, Multiplexer_0_out, clk);
  assign const_0=4'b0101;
  Random #(4) Random_0(Random_0_RandomValue, clk, const_0);

  always @ (Random_0_RandomValue)
    $display("Random_0_RandomValue = %d", Random_0_RandomValue);

endmodule

module Multiplexer8(out, in0, in1, in2, in3, in4, in5, in6, in7, sel);
  parameter WIDTH = 1;
  output reg [WIDTH-1:0] out;
  input [WIDTH-1:0] in0, in1, in2, in3, in4, in5, in6, in7;
  input [2:0] sel;

  always @ (*)
    case (sel)
      0 : out = in0;
      1 : out = in1;
      2 : out = in2;
      3 : out = in3;
      4 : out = in4;
      5 : out = in5;
      6 : out = in6;
      7 : out = in7;
    endcase
endmodule

module Multiplexer2(out, in0, in1, sel);
  parameter WIDTH = 1;
  output reg [WIDTH-1:0] out;
  input [WIDTH-1:0] in0, in1;
  input [0:0] sel;

  always @ (*)
    case (sel)
      0 : out = in0;
      1 : out = in1;
    endcase
endmodule

module TflipFlop(q, q_inv, clk, t, a_rst, pre, en);
  parameter WIDTH = 1;
  output reg [WIDTH-1:0] q, q_inv;
  input clk, a_rst, pre, en;
  input [WIDTH-1:0] t;

  always @ (posedge clk or posedge a_rst)
    if (a_rst) begin
      q <= 'b0;
      q_inv <= 'b1;
    end else if (en == 0) ;
    else if (t) begin
      q <= q ^ t;
      q_inv <= ~q ^ t;
    end
endmodule

module DflipFlop(q, q_inv, clk, d, a_rst, pre, en);
  parameter WIDTH = 1;
  output reg [WIDTH-1:0] q, q_inv;
  input clk, a_rst, pre, en;
  input [WIDTH-1:0] d;

  always @ (posedge clk or posedge a_rst)
    if (a_rst) begin
      q <= 'b0;
      q_inv <= 'b1;
    end else if (en == 0) ;
    else begin
      q <= d;
      q_inv <= ~d;
    end
endmodule

module Demultiplexer2(out0, out1, in, sel);
  parameter WIDTH = 1;
  output reg [WIDTH-1:0] out0, out1;
  input [WIDTH-1:0] in;
  input [0:0] sel;

  always @ (*) begin
    out0 = 0;
    out1 = 0;
    case (sel)
      0 : out0 = in;
      1 : out1 = in;
    endcase
  end
endmodule

module PriorityEncoder2(sel, ze, in0, in1);
  output reg [0:0] sel;
  output reg ze;
  input in0, in1;

  always @ (*) begin
    sel = 0;
    ze = 0;
    if (in1)
      sel = 1;
    else if (in0)
      sel = 0;
    else
      ze = 1;
  end
endmodule

module Decoder2(out0, out1, sel);
  output reg out0, out1;
  input [0:0] sel;

  always @ (*) begin
    out0 = 0;
    out1 = 0;
    case (sel)
      0 : out0 = 1;
      1 : out1 = 1;
    endcase
  end
endmodule

module Random(val, clk, max);
  parameter WIDTH = 1;
  output reg [WIDTH-1:0] val;
  input clk;
  input [WIDTH-1:0] max;

  always @ (posedge clk)
    val = $urandom_range(0, max);
endmodule

Note: Please check Allow edits from maintainers. if you would like us to assist in the PR.

Coverage decreased (-0.06%) to 88.045% when pulling a1fa67488eccca56237b837b2de4a1b5fa9e7936 on drjyeh:master into 87c6e1c2356f1f8ee7085a4edcd5bd937cc3b609 on CircuitVerse:master.

Instead of having display statements in DigitalLed.generateVerilog() can we have it in  DigitalLed.moduleVerilog(). It seems neater as the user can replace the DigitalLed verilog module with his/her custom logic easily then?

Hi,
Yes I think this is a great idea.
We need to make sure we can write the pin name somehow in moduleVerilog because it is not a prototype function.
Thanks,
James
James H-J Yeh, Ph.D.
Associate Professor, Engineering and Computer Science
Azusa Pacific University

On Oct 15, 2020, at 12:36 PM, Satvik Ramaprasad<EMAIL_ADDRESS>wrote:
Instead of having display statements in DigitalLed.generateVerilog() can we have it in DigitalLed.moduleVerilog(). It seems neater as the user can replace the DigitalLed verilog module with his/her custom logic easily then?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/CircuitVerse/CircuitVerse/pull/1809#issuecomment-709547331, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMS2ORXE6Q6DQQPQTDZHCV3SK5FNTANCNFSM4SQ5ZGIQ.

Hi, Yes I think this is a great idea. We need to make sure we can write the pin name somehow in moduleVerilog because it is not a prototype function. Thanks, James --- James H-J Yeh, Ph.D. Associate Professor, Engineering and Computer Science Azusa Pacific University
…
On Oct 15, 2020, at 12:36 PM, Satvik Ramaprasad @.***> wrote: Instead of having display statements in DigitalLed.generateVerilog() can we have it in DigitalLed.moduleVerilog(). It seems neater as the user can replace the DigitalLed verilog module with his/her custom logic easily then? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1809 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMS2ORXE6Q6DQQPQTDZHCV3SK5FNTANCNFSM4SQ5ZGIQ.

Ohh I didn't think of this :(, I guess this would be a problem. The problem is not actually not due to the prototype but because the same verilog module cannot display different names. Can Verilog modules take string as a parameter? Then we can pass the name of the string also in generateVerilog() code and then simply use the string name inside code in moduleVerilog.

parameter WIDTH = 1;
Have we hardcoded width?

I check on this and Verilog cannot take names as parameters.  Sorry we may have to keep it is for now.
James H-J Yeh, Ph.D.
Associate Professor, Engineering and Computer Science
Azusa Pacific University

On Oct 17, 2020, at 4:58 AM, Satvik Ramaprasad<EMAIL_ADDRESS>wrote:
Hi, Yes I think this is a great idea. We need to make sure we can write the pin name somehow in moduleVerilog because it is not a prototype function. Thanks, James --- James H-J Yeh, Ph.D. Associate Professor, Engineering and Computer Science Azusa Pacific University
… x-msg://39/#
On Oct 15, 2020, at 12:36 PM, Satvik Ramaprasad @.***> wrote: Instead of having display statements in DigitalLed.generateVerilog() can we have it in DigitalLed.moduleVerilog(). It seems neater as the user can replace the DigitalLed verilog module with his/her custom logic easily then? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1809 (comment) https://github.com/CircuitVerse/CircuitVerse/pull/1809#issuecomment-709547331>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMS2ORXE6Q6DQQPQTDZHCV3SK5FNTANCNFSM4SQ5ZGIQ https://github.com/notifications/unsubscribe-auth/AMS2ORXE6Q6DQQPQTDZHCV3SK5FNTANCNFSM4SQ5ZGIQ.
Ohh I didn't think of this :(, I guess this would be a problem. The problem is not actually not due to the prototype but because the same verilog module cannot display different names. Can Verilog modules take string as a parameter? Then we can pass the name of the string also in generateVerilog() code and then simply use the string name inside code in moduleVerilog.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/CircuitVerse/CircuitVerse/pull/1809#issuecomment-710864476, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMS2ORUXYEJG2NGTTT6IAC3SLGBGBANCNFSM4SQ5ZGIQ.

For Mux and Demux the WDITH parameter gets overridden when they are instantiated.
James H-J Yeh, Ph.D.
Associate Professor, Engineering and Computer Science
Azusa Pacific University

On Oct 17, 2020, at 5:47 AM, Satvik Ramaprasad<EMAIL_ADDRESS>wrote:
parameter WIDTH = 1;
Have we hardcoded width?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/CircuitVerse/CircuitVerse/pull/1809#issuecomment-710912467, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMS2ORX3PF55L7LRYBLLWB3SLGG6XANCNFSM4SQ5ZGIQ.


add prefix to makefile

This would allow for make --PREFIX=/usr install and so on.

Thank you very much! Merged withouth further ado..


404 at /examples/running_ssh_service/

I found a broken link : https://docs.docker.com/examples/running_ssh_service/
I arrived on this page through: https://github.com/eg-cert/dockpot

Hey @javrong
This page is no longer available. Please see the link here: https://docs.docker.com/samples/running_ssh_service/.


don't hit sig/get if we don't have to

the merkle tree might say our cached copy is up-to-date, and we would have been hitting the sig/get endpoint just to get a nil update, so eliminate the RT

cc @mmaxim


Move is_same_XXX methods to __eq__

Closes #75

thanks for this @nmheim

This is ready to be merged IMO :)

@nmheim What is preventing from merging this one?

This is ready to be merged IMO :)

Yes, I would go ahead.


Expose well-known certificate subject fields in Lua filter, such as CN and O.

Title: Expose well-known certificate subject fields in Lua filter, such as CN and O.
Description:
Today, Lua code can access certificate subject in RFC 2253 format via connection():ssl():subjectPeerCertificate() (doc). However, the subject string is kind of useless for code, because usually what we need is checking some field value (such as CN) in the subject. And parsing RFC 2253 string is pretty tricky (just read RFC4514 and RFC2253 will get the idea). People often use a simple regex to extract the field value which can be easily attacked by malicious certificate with escaped characters and put the service into security risk.
So I think it's better to expose the fields value from subject in Lua API directly, since internally envoy already parsed the certificate from ASN.1 format into data structures, directly return those fields actually avoid both generating RFC2253 and parsing RFC2253 string.
And we probably don't need expose every fields (since some ASN.1 type just can not be converted to string), just some well-known ones are good enough for most user scenarios. The purpose is not creating a strict X509 Name object, but reducing user's extra logic and security risk to access these common subject fields. A good example is the golang's pkix Name implementation, which exposed these fields:
	"<IP_ADDRESS>":  "C",
	"<IP_ADDRESS>": "O",
	"<IP_ADDRESS>": "OU",
	"<IP_ADDRESS>":  "CN",
	"<IP_ADDRESS>":  "SERIALNUMBER",
	"<IP_ADDRESS>":  "L",
	"<IP_ADDRESS>":  "ST",
	"<IP_ADDRESS>":  "STREET",
	"<IP_ADDRESS>": "POSTALCODE",

Relevant Links:
RFC2253
RFC4514

cc @wbpcode

@ggreenway @wbpcode could one of you take a look here and see if the code changes @zhishi proposed look reasonable to you? I think this is a valid use-case

@zhishi can you post a PR adding one of the accessors you're planning? We can evaluate the API better in the context of the code needed to implement it.

thanks for the response, currently you can see the code change here
https://github.com/envoyproxy/envoy/compare/main...zhishi:envoy:zhimengshi/subject_rdn
although it works, but I need to add unit tests to make it a ready PR. I hope this can provide enough context of the code change.

Not stale


Update specificity of custom CSS rules and added another header example

There's a chance that the recent CodeMirror CSS change broke most of the Tips (tested for example with the header example) so I think it makes sense to update the rules with the new specificity.
I've also added the unindented header example from Uxamanda's post as I couldn't find it in your doc (although it appears in the screenshot).

Good point, indeed it looks like it did break some of the tips. Thanks for the update!