Probing Leaf..

Medium posts:

Rust Guide to CHANGELOG.md
About Rust’s Machine Learning Community
Increasing Open-Source Engagement w/ Structural Communication Guidelines for Code Documentation
Tensorflow wins

On automatic differentiation (AD) and optimizations:
gitter:

Michael Hirn: We are currently working on the backend abstraction and parallelisation for CPU and GPU support.
We were looking into auto diff a bit already.
I will link quickly to the reddit comments about auto diff on leaf: discussion
auto diff would be nice, but it comes with computational cost, which we can avoid in plain ML scenarios. So not a super high prio at the moment.
Dominique Luna: I was thinking something along the lines of repo and repo rather than Theano. I think Tensorflow is somewhere in between.
As for performance, you're right it comes at a cost but according to this post  it's typically only 2x and they still have optimizations they can make. Also, if you're going to consider the computer time you also should consider the developer's time should is worth more and auto diff helps tremendously.
....
MH: ..will receive a hard refactor to support the backend abstraction (mem synchronization) and then of course the new backend crate to support executing the computations across platforms and devices..

Declarative (Tensorflow, Theano, etc.) vs Imperative (Leaf, Torch, etc.) approaches
hn:

I think CNTK and Tensorflow and Theano, too, have a declarative approach, representing the computation via a computational graph. Which in my opinion is beneficial for research. But for a hacker or a software developer who wants to build an application, this creates an unnecessarily steep learning curve and feels unintuitive. (I have the feeling, that this an important reason why Keras, Lasagne and co. exists)
Leaf takes an imperative approach and explores an easier API (only Layers Functions and Solvers (Optimizer Algorithms)), reusability through modularity and abstractions that keep the implementation and concepts to a minimum or rather abstractions that feel as familiar to a hacker as possible.
For future versions e.g., we want to explore what is practically possible with auto-differentiation via dual numbers and differentiable programming.

Solvers:

We wanted to make clear that we are not "just" doing deep learning with Leaf. A lot of other ML algorithms that don't fall under the deep learning label can benefit of the tools provided (e.g. solvers) and can be implemented as layers. While not too popular, there are papers showing this being done with e.g. a SVM.

hn:

rryan: Great work -- very exciting to see a Rust ML framework!
Dist-Belief (TensorFlow's predecessor) was oriented around layers (as Leaf is) but TensorFlow (like Theano) is graph/flow-based rather than layer-based. I think in particular the layer-based approach makes it awkward to represent more complicated models since not every model can be decomposed into backward/forward passes.
Do you worry that by adopting a layer-based paradigm you are boxing yourself in? A graph-based approach does seem more flexible (and distributable across machines).
M. Goisser: > I am very optimistic concerning our plans to make Leaf work in a distributed fashion. It's not quite finalized yet, but the rough outline consists of abstracting other parts of the Network away as a special type of layer and using capnproto + zeromq for transportation. Sadly I haven't come across a good implementation of model parallelism to learn from, so that might become a bit tricky.
I think there is still a lot of room for improvement for layer-based networks, but even if the graph-based approach will become more prelevant I think the Rust trait system gives us great tools to migrate and reuse a lot from our layer based approach.
Djvulee: I just curious about is there any good Actor model library in Rust, just like Akka for Scala/Java, so there is a better model for transportation.

Medium posts:

Rust Guide to CHANGELOG.md
About Rust’s Machine Learning Community
Increasing Open-Source Engagement w/ Structural Communication Guidelines for Code Documentation
Tensorflow wins

On automatic differentiation (AD) and optimizations:
gitter:

Michael Hirn: We are currently working on the backend abstraction and parallelisation for CPU and GPU support.
We were looking into auto diff a bit already.
I will link quickly to the reddit comments about auto diff on leaf: discussion
auto diff would be nice, but it comes with computational cost, which we can avoid in plain ML scenarios. So not a super high prio at the moment.
Dominique Luna: I was thinking something along the lines of repo and repo rather than Theano. I think Tensorflow is somewhere in between.
As for performance, you're right it comes at a cost but according to this post  it's typically only 2x and they still have optimizations they can make. Also, if you're going to consider the computer time you also should consider the developer's time should is worth more and auto diff helps tremendously.
....
MH: ..will receive a hard refactor to support the backend abstraction (mem synchronization) and then of course the new backend crate to support executing the computations across platforms and devices..

Declarative (Tensorflow, Theano, etc.) vs Imperative (Leaf, Torch, etc.) approaches
hn:

I think CNTK and Tensorflow and Theano, too, have a declarative approach, representing the computation via a computational graph. Which in my opinion is beneficial for research. But for a hacker or a software developer who wants to build an application, this creates an unnecessarily steep learning curve and feels unintuitive. (I have the feeling, that this an important reason why Keras, Lasagne and co. exists)
Leaf takes an imperative approach and explores an easier API (only Layers Functions and Solvers (Optimizer Algorithms)), reusability through modularity and abstractions that keep the implementation and concepts to a minimum or rather abstractions that feel as familiar to a hacker as possible.
For future versions e.g., we want to explore what is practically possible with auto-differentiation via dual numbers and differentiable programming.

Solvers:

We wanted to make clear that we are not "just" doing deep learning with Leaf. A lot of other ML algorithms that don't fall under the deep learning label can benefit of the tools provided (e.g. solvers) and can be implemented as layers. While not too popular, there are papers showing this being done with e.g. a SVM.

hn:

rryan: Great work -- very exciting to see a Rust ML framework!
Dist-Belief (TensorFlow's predecessor) was oriented around layers (as Leaf is) but TensorFlow (like Theano) is graph/flow-based rather than layer-based. I think in particular the layer-based approach makes it awkward to represent more complicated models since not every model can be decomposed into backward/forward passes.
Do you worry that by adopting a layer-based paradigm you are boxing yourself in? A graph-based approach does seem more flexible (and distributable across machines).
M. Goisser: > I am very optimistic concerning our plans to make Leaf work in a distributed fashion. It's not quite finalized yet, but the rough outline consists of abstracting other parts of the Network away as a special type of layer and using capnproto + zeromq for transportation. Sadly I haven't come across a good implementation of model parallelism to learn from, so that might become a bit tricky.
I think there is still a lot of room for improvement for layer-based networks, but even if the graph-based approach will become more prelevant I think the Rust trait system gives us great tools to migrate and reuse a lot from our layer based approach.
Djvulee: I just curious about is there any good Actor model library in Rust, just like Akka for Scala/Java, so there is a better model for transportation.

Non-sequential containers
gitter:

M. Goisser: You can get pretty far in composing a directed graph with sequential layers alone, and in my opinion that should be the main Leaf interface. There are however a few utility layers that are missing and would allow for more options.
For example a Split layer.
There is no Function layer, and I think that that would only really work on the CPU.
I feel like you are looking for a more symbolic aproach than what Leaf offers?

Alexander Morozov comments
gitter:

About roadmap I'd suggest 1) merge most current PRs, they are mostly approved, just stalled, 2) cleanup Leaf code, there is a lot of boilerplate, е. g. I think Arc<RwLocks<>> are not actually required, but I'm not completely sure, 3) fix GPU underutilization issues: published benchmarks AFAIK test just forward/backward passes without I/o and solver parts. When they are added, Leaf is not so fast - nvidia-smi showed that GPU utilization is about 40% on my low-mid range card. I've submitted several PRs that reduce CUDA i/o and increase performance 10-30%, and I'm sure a lot more can be done.
About (3) -- I've got some unbased speculations that "imperative-style" approach will eventually limit possible performance, as it would be difficult to submit several commands to GPU at once. It's certainly possible, but it will increase code complexity, while Leaf goal was to keep code as simple and hackable as possible. @/MichaelHirn didn't agree with me on this and he definitely understands NNs and GPUs better, so I think best way to find out is to take time to profile a complete NN with solver and find out what bottlenecks currently are there and if they can be removed for wide enough range of networks.

Supporting the creation of dynamic computation graphs (DCG) or static computation graphs (SCG)
Todo..


[tool_crash] ProcessException: Operation did not complete successfully because the file contains a virus or potentially unwanted software. Command: C:\dev\flutter\bin\cache\dart-sdk\bin\dartaotruntime.exe, OS error code: 225

Command
flutter --no-color run --machine --track-widget-creation --device-id=emulator-5554 --start-paused --dart-define=flutter.inspector.structuredErrors=true lib\main.dart

Steps to Reproduce

...
...
...

Logs
ProcessException: Operation did not complete successfully because the file contains a virus or potentially unwanted software.
Command: C:\dev\flutter\bin\cache\dart-sdk\bin\dartaotruntime.exe, OS error code: 225
#0      _ProcessImpl._start (dart:io-patch/process_patch.dart:402:33)
#1      Process.start (dart:io-patch/process_patch.dart:38:20)
#2      LocalProcessManager.start (package:process/src/interface/local_process_manager.dart:42:22)
#3      ErrorHandlingProcessManager.start.<anonymous closure> (package:flutter_tools/src/base/error_handling_io.dart:686:24)
#4      _run (package:flutter_tools/src/base/error_handling_io.dart:556:20)
#5      ErrorHandlingProcessManager.start (package:flutter_tools/src/base/error_handling_io.dart:685:12)
#6      DefaultResidentCompiler._compile (package:flutter_tools/src/compile.dart:874:37)
#7      DefaultResidentCompiler._recompile (package:flutter_tools/src/compile.dart:734:14)
#8      _RecompileRequest._run (package:flutter_tools/src/compile.dart:420:16)
#9      _CompilationRequest.run (package:flutter_tools/src/compile.dart:394:30)
#10     DefaultResidentCompiler._handleCompilationRequest (package:flutter_tools/src/compile.dart:780:23)
#11     _rootRunUnary (dart:async/zone.dart:1407:47)
#12     _CustomZone.runUnary (dart:async/zone.dart:1308:19)
#13     _CustomZone.runUnaryGuarded (dart:async/zone.dart:1217:7)
#14     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)
#15     _DelayedData.perform (dart:async/stream_impl.dart:515:14)
#16     _PendingEvents.handleNext (dart:async/stream_impl.dart:620:11)
#17     _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:591:7)
#18     _rootRun (dart:async/zone.dart:1391:47)
#19     _CustomZone.run (dart:async/zone.dart:1301:19)
#20     _CustomZone.runGuarded (dart:async/zone.dart:1209:7)
#21     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1249:23)
#22     _rootRun (dart:async/zone.dart:1399:13)
#23     _CustomZone.run (dart:async/zone.dart:1301:19)
#24     _CustomZone.runGuarded (dart:async/zone.dart:1209:7)

[✓] Flutter (Channel stable, 3.19.5, on Microsoft Windows [Version 10.0.22000.2538], locale en-US)
    • Flutter version 3.19.5 on channel stable at C:\dev\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 300451adae (5 weeks ago), 2024-03-27 21:54:07 -0500
    • Engine revision e76c956498
    • Dart version 3.3.3
    • DevTools version 2.31.1

[✓] Windows Version (Installed version of Windows is version 10 or higher)

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at C:\Users\HP\AppData\Local\Android\sdk
    • Platform android-UpsideDownCakePrivacySandbox, build-tools 34.0.0
    • Java binary at: C:\Program Files\Android\Android Studio\jbr\bin\java
    • Java version OpenJDK Runtime Environment (build 17.0.9+0--11185874)
    • All Android licenses accepted.

[✓] Chrome - develop for the web
    • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[✗] Visual Studio - develop Windows apps
    ✗ Visual Studio not installed; this is necessary to develop Windows apps.
      Download at https://visualstudio.microsoft.com/downloads/.
      Please install the "Desktop development with C++" workload, including all of its default components

[✓] Android Studio (version 2023.2)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.9+0--11185874)

[✓] VS Code (version 1.88.1)
    • VS Code at C:\Users\HP\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension can be installed from:
      🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected device (4 available)
    • AOSP on IA Emulator (mobile) • emulator-5554 • android-x86    • Android 9 (API 28) (emulator)
    • Windows (desktop)            • windows       • windows-x64    • Microsoft Windows [Version 10.0.22000.2538]
    • Chrome (web)                 • chrome        • web-javascript • Google Chrome 124.0.6367.63
    • Edge (web)                   • edge          • web-javascript • Microsoft Edge 124.0.2478.67

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

Flutter Application Metadata
Type: app
Version: 1.0.0+1
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: stable
Creation framework version: 300451adae589accbece3490f4396f10bdf15e6e

Closing this as a duplicate of https://github.com/flutter/flutter/issues/61997


fix: make Influxdb lines able to be inserted into last created tables

I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
In #3794 we made a breaking change that Influxdb lines are all parsed as nanosecond precision timestamp, which let the Influxdb tables created before that PR unable to be put continued. This PR is trying to fix that breaking change.
As an instance, first create a table simulating an old influxdb table with the previous default millisecond precision:
CREATE TABLE IF NOT EXISTS `monitor` (
  `host` STRING NULL,
  `cpu` DOUBLE NULL,
  `memory` DOUBLE NULL,
  `ts` TIMESTAMP NOT NULL, # intentionally use millisecond precision
  TIME INDEX (`ts`),
  PRIMARY KEY (`host`)
)

Before this PR, put influxdb lines will result in error:
curl -i -XPOST "http://localhost:4000/v1/influxdb/write?db=public&precision=ms" \
--data-binary \
'monitor,host=<IP_ADDRESS> cpu=0.1,memory=0.4<PHONE_NUMBER>450
 monitor,host=<IP_ADDRESS> cpu=0.2,memory=0.3<PHONE_NUMBER>450
 monitor,host=<IP_ADDRESS> cpu=0.5,memory=0.2<PHONE_NUMBER>450'
HTTP/1.1 500 Internal Server Error
content-type: application/json
content-length: 173
date: Fri, 28 Jun 2024 04:34:38 GMT

{"error":"Invalid request to region<PHONE_NUMBER>400(1025, 0), reason: column ts expect type Timestamp(Millisecond(TimestampMillisecondType)), given: TIMESTAMP_NANOSECOND(18)"}%

After this PR, it's ok to put again:
curl -i -XPOST "http://localhost:4000/v1/influxdb/write?db=public&precision=ms" \
--data-binary \
'monitor,host=<IP_ADDRESS> cpu=0.1,memory=0.4<PHONE_NUMBER>450
 monitor,host=<IP_ADDRESS> cpu=0.2,memory=0.3<PHONE_NUMBER>450
 monitor,host=<IP_ADDRESS> cpu=0.5,memory=0.2<PHONE_NUMBER>450'
HTTP/1.1 204 No Content
date: Fri, 28 Jun 2024 04:41:40 GMT

Checklist

[ ] I have written the necessary rustdoc comments.
[ ] I have added the necessary unit tests and integration tests.
[ ] This PR requires documentation updates.

Summary by CodeRabbit

New Features

Enhanced InfluxDB request handling by supporting asynchronous row insert requests.
Improved timestamp handling with refined precision and error management.

Refactor

Updated timestamp units to use TimeUnit instead of TimestampType.
Centralized test utility for InfluxDB tests to support both standalone and distributed instances.

Tests

Refactored tests to utilize the new common test utility for InfluxDB line insertion with varying time precision levels.

@MichaelScofield It's better to use an option to control this behavior and it's disabled by default.

@killme2008 The current behavior is unaffected, new influxdb tables are still nanosecond time index. This PR simply brings compatibility back, so I don't think there's a need for an option.

@killme2008 The current behavior is unaffected, new influxdb tables are still nanosecond time index. This PR simply brings compatibility back, so I don't think there's a need for an option.

Alignment and checking typically impact performance, often unnecessary.
In other words, the user may create a table with a millisecond time index, and ingestion with influxdb, which we want to prohibit.

We already discussed this before and have an agreement.  Maybe you can discuss it directly with @fengjiachun . I have to sleep.

@killme2008 I have added an option for that.


Remove Fn1?

It's just (->), right?

Yeah, it's exactly that. I guess it was just added for symmetry with the rest. I'm fine with getting rid of it, @paf31?

Alternatively, we could perhaps define a type synonym.


Add Bran Cowan to contributors list

I am adding my name to contributors list

I realized that the nickname I used doesn't match what's on Canvas so I had to fix that.


about UNK and hits_at_k

I'm doing a classification task. in my ludwig model's result, there is one more class appear, the UNK class. could anyone explain what is UNK?
moreover, what is hits_at_k?

UNK is a sink class that deals with all the cases where at test time you have a class that was not present at training time. You can also set a threshold on the prequency of classes and all the ones less frequent than a certain amount will be replaced with UNK.
Hits at k is the accuracy considering any prediction in the first k to be correct instead of the argmax. It is useful if you have a model that fuels more than one prediction to a UI, your model behaves well if any of the k predictions is correct.


Add new Shipping Option to discount shipping

Part 1 - Coding:
Add the new shipping option to discount shipping.

Part 2 - Design Shipping App to use the Unit of Work pattern via commit: 46680d0


Token repeat limit, hallucination helper option.

Implemented an argument option to transcribe token_repeat_limit .
Predictions with token repeats (consecutive) over this, are failed (with fallback, like other thresholds)
Implemented with filtering of simple extreme repetition (100+ tokens) in mind.
By default this option is disabled of course, and doesn't go inside the if branch added.

 Hmmm...
 I don't know...
 Like, like...

The "Hmmmm..." prediction goes so long for the prediction with many repeats that it eats up the following other dialogue for some reason (segment from 2.0s to 13.9s, when the Hmmm... actually takes around 1.82 seconds).
I've seen similar repeats happen multiple times in my experience with different audio tracks, but if you want the sample I can share.
Kind of specific, but also very simple addition.
A token_repeat_size could be implemented for cases where tokens are repeated like [x,y, x,y, x,y, ... ], instead of just [x,x,x...]

compression_ratio_threshold already triggers a fallback when there are too many repetitions. Why do we need a new option for that?

compression_ratio_threshold already triggers a fallback when there are too many repetitions. Why do we need a new option for that?

compression_ratio_threshold: If the gzip compression ratio is above this value, treat as failed.

No wonder I didn't find this option to deal with repetition. Thanks.
Is there an intuitive way to decide on a value for the compression_ratio_threshold, for the example I gave of obviously extreme repetitions (100+)?
Makes me wonder why a reasonably high value for this isn't the default.

Could you share the audio sample?

https://files.catbox.moe/m8jj9r.wav
tests were with the medium model, it does get the transcription right most of the time.
Would any compression_ratio_threshold make sure 100 token repeats aren't the output?

I don't get any "Hmmm....":
With VAD:
[00:00.000 --> 00:01.700]  Do you have any suggestions?

No VAD:
[00:00.000 --> 00:01.920]  Do you have any suggestions?
[00:05.420 --> 00:06.820]  I don't know...
[00:06.820 --> 00:07.580]  Like...like...
[00:07.580 --> 00:08.320]  Fuwawa desu!
[00:08.420 --> 00:09.180]  Wa wa wa wa!
[00:09.520 --> 00:10.260]  Mokoko desu!
[00:10.400 --> 00:11.160]  Co co co co!
[00:11.560 --> 00:12.260]  Fuwa mo ko desu!
[00:12.400 --> 00:13.020]  Wa wa wa wa!

I used my cli version, it has different defaults.

I didn't set compression_ratio_threshold but it let that compression_ratio=31 pass...? (and others on repeated tests) Is something else making it ignore the compression_ratio_threshold? On my tests it happened on both 0 and non-0 temp segments (0.0, 0.2).
Tested again to make sure you can see in TranscriptionInfo printed below compression_ratio_threshold=2.4, but the result had compression_ratio=31. Maybe I'm misunderstanding. (as the normal looking "Do you have any suggestions?" segment pred has compression_ratio=10)

If you set the compression_ratio_threshold to 2.4, fallback will be triggered for segments that exceed it. And after fallback, it will try to choose the best result. If a sentence with a compression_ratio of 30 looks like the "best" result, it will be chosen.
So if you want to limit the results based on compression_ratio:

manually filter sentences from the final results

try this alternative, which will probably likely to give you better results.
https://github.com/guillaumekln/faster-whisper/discussions/371

this PR will set avg_logprob and compression_ratio limits for the final result based on log_prob_threshold and compression_ratio_threshold.
https://github.com/guillaumekln/faster-whisper/discussions/371#discussioncomment-6540179

If a sentence with a compression_ratio of 31 looks like the "best" result, that will be selected, even if it has a very high compression_ratio value. The same is true for log_prob_threshold.

gah, and there's no way to override this I suppose. that's a bit sad. thanks

I don't get any "Hmmm....":

I does only happen rarely, as I said it works most of the time; I was unlucky to get it too many times for my liking

Changing the length_penalty value can help, but it can affect the result.
https://github.com/guillaumekln/faster-whisper/discussions/371 You can also try this.


Try and make loading GT a little bit faster

Right now loading gt takes about:
real	25m37.033s
user	19m6.876s
sys	1m21.867s

reduced to
real	16m29.988s user	12m32.679s sys	1m42.550s
results may vary :)


Error Code Handling

Is it possible to get the error code that's returned in the .error callback?

If it is an NSError, then it is has a code property. If it is not an NSError, then it depends on its type.

foo.catch { err in
    let code = (err as NSError).code
}

Thanks for the quick reply and all the work you've done Max.
The issue that I'm facing is that the error that is being rejected from my Database authenticate method is of type ErrorType, while the error response result that I'm receiving in the Database authenticate method (before it is rejected) is of type NSError.
Any ideas as to why this might be happening?
struct Database {
    static func authenticate(facebookUID: String) -> Promise<Token> {
        let properties = NSMutableDictionary()
        properties.setValue(facebookUID, forKeyPath: "facebookUID")

        let prefs = NSUserDefaults.standardUserDefaults()
        if let deviceToken = prefs.objectForKey("deviceToken") as? NSData {
            let token = deviceToken.hexString
            properties.setValue(token, forKeyPath: "deviceToken")
        }

        return Promise { fulfill, reject in
            Alamofire.request(Router.Authenticate((properties as NSDictionary) as! [String : AnyObject])).validate()
                .responseObject { (response: Response<Token, NSError>) in
                    switch response.result {
                    case .Success(let token):
                        fulfill(token)
                    case .Failure(let error):
                        // error is of type NSError
                        print("authenticate error")
                        reject(error)
                    }
            }
        }
    }
}

class SomeOtherClass {
    func authenticate(user: User) {
        guard let facebookUID = user.facebookUID else {
            return
        }

        Database.authenticate(facebookUID).then { token -> Promise<User> in
            Router.OAuthToken = token
            return Database.getCurrentUser()
        }.then { currentUser -> Void in
            let prefs = NSUserDefaults.standardUserDefaults()
            prefs.setObject(NSKeyedArchiver.archivedDataWithRootObject(currentUser), forKey: "currentUser")
            let containerViewController = ContainerViewController()
            self.presentViewController(containerViewController, animated: false, completion: nil)
        }.error { err -> Void in
            // err is of type ErrorType
        }
    }
}

Thanks for the help!

Is there a reason that when I reject an error of type NSError, in my .error callback, it's of type ErrorType and I have to convert it over to NSError to use it?

ErrorType is the Swift error type, so that's what you get.
The only way we could do otherwise is to have there error type be generic too, but this would make it impossible to have different promises with different error types be chained without forcing the user to provide some kind of glue code every time a promise is chained.


[sw, test, otbn] Add the chip_sw_otbn_op test

This PR :
Add the test chip_sw_otbn_op:

SW test directs the OTBN engine to perform an ECDSA operation.
SW validates the reception of the otbn done interrupt once the operation is complete.
SW verifies the correctness of the result with the expected value which is pre-computed using a reference model.

Add an interrupt service to opentitan/sw/device/lib/runtime/otbn.c

Add function otbn_busy_wait_for_done_irq.
Refactor the function otbn_init to receive the plic base address.
Add new fields to the otbn_t struct
Move the irq implementation from otbn_irq_test.c to otbn.c.

Just to make sure I understand the context -- the goal here is that instead of constantly polling the OTBN status bits as in otbn_busy_wait_for_done, we should wait for OTBN to offer its "done" interrupt, because that will improve performance? If so, does it actually have a performance impact in wall-clock time, given that we have to wait for OTBN either way?
One piece of information that seems relevant here: the sw/device/lib/runtime/otbn.c driver is one of three very similar but subtly different OTBN drivers currently in the repository:

sw/device/lib/runtime/otbn.c is based on the OTBN DIF and only used for hardware testing (since the DIFs are not supposed to be used for other purposes, iiuc).
sw/device/silicon_creator/lib/otbn_util.c is not DIF-based and is exclusively for the mask ROM/ROM_EXT (e.g. for sigverify).
sw/device/lib/crypto/otbn_util.c is not DIF-based and is used for the crypto library; it's almost identical to the silicon_creator one but with different base libraries.

All three drivers have otbn_busy_wait_for_done, otbn_t`, etc., so it's possible that changes will need to be replicated to the other drivers -- it depends on why exactly these changes are needed, and whether they're relevant to purposes other than hardware testing.

Just to make sure I understand the context -- the goal here is that instead of constantly polling the OTBN status bits as in otbn_busy_wait_for_done, we should wait for OTBN to initiate its "done" interrupt, because that will improve performance? If so, does it actually have a performance impact in wall-clock time, given that we have to wait for OTBN either way?
One piece of information that seems relevant here: the sw/device/lib/runtime/otbn.c driver is one of three very similar but subtly different OTBN drivers currently in the repository:

sw/device/lib/runtime/otbn.c is based on the OTBN DIF and only used for hardware testing (since the DIFs are not supposed to be used for other purposes, iiuc).
sw/device/silicon_creator/lib/otbn_util.c is not DIF-based and is exclusively for the mask ROM/ROM_EXT (e.g. for sigverify).
sw/device/lib/crypto/otbn_util.c is not DIF-based and is used for the crypto library; it's almost identical to the silicon_creator one but with different base libraries.

All three drivers have otbn_busy_wait_for_done, otbn_t, etc., so it's possible that changes will need to be replicated to the other drivers -- it depends on why exactly these changes are needed, and whether they're relevant to purposes other than hardware testing.

Thanks @jadephilipoom for the review.
That's A good question, I did the change not because of performance but to match the test plan.
Using irq will only improve performance if the main core (ibex) can do something else other than wait for the otbn to finish. So in that sense, even the new otbn_busy_wait_for_done_irq function will not improve performance as it blocks as well.
Summarizing, this refactoring just applies to the sw/device/lib/runtime/otbn.c driver as it does not improve performance, but just test the irq itself.


Symbols for Markdown Heading Can Be Refined

Currently heading of markdown is displayed as 'functions'

Currently bullet point of markdown  is not displayed

+1
Hope there will be different indentations based heading level too

@gudh the improvement proposed by @liuderchi has been added in the forked package symbols-tree-nav.


CoreDNS is burning CPU and spamming my network with DNS requests to <IP_ADDRESS>

Sorry I don't know what logs to capture to help pinpoint the issue more precisely, so please let me know if you'd like me to capture appropriate diagnostics.
I noticed that my Home Assistant OS running in a VM on an Intel NUC was using a lot more CPU than I remembered.  And after logging in via SSH, I saw that the coredns process is using anywhere from 40-60% of the virtual CPU and had burned over 900 hours of processing time since its last restart!
As it happens, my router has been having connectivity issues lately so on a hunch I ran a tcpdump.  There I found a constant endless stream of DNS requests to Cloudflare's <IP_ADDRESS> DNS service originating from the server that hosts my Home Assistant instance.
15:53:55.887610 IP XXX.lan.50080 > one.one.one.one.853: Flags [S], seq<PHONE_NUMBER>, win 64240, options [mss 1460,sackOK,TS val<PHONE_NUMBER> ecr 0,nop,wscale 7], length 0
15:53:55.887936 IP one.one.one.one.853 > XXX.lan.50080: Flags [R.], seq 0, ack<PHONE_NUMBER>, win 0, length 0

My router is configured to hijack DNS requests from the local network for privacy reasons.  When I disable that feature, the spam stops and the excess CPU usage vanishes.  So Home Assistant seems not to like the responses that it is receiving and it just keeps trying again and again, flooding the network.
Why is Home Assistant attempting to bypass my locally configured DNS server?  And why is it retrying so aggressively?
Home Assistant Version: core-2022.5.0
Supervisor Version: supervisor-2022.05.0

Ok, I see in other recently closed issues that the requests to Cloudflare's DNS service are intended behavior to implement DNS over TLS.  As it happens, my router is already configured to take care of that for all clients on the local network (hence the DNS hijacking).
It's not clear to me whether the bug that causes the network to be flooded with requests has been fixed though.  That was getting close to DoSing my network's DNS server from the inside.

So I actually don't know and its bothering me. It shouldn't be doing this and I've made a note to look into it. It almost seems like blocking the fallback DNS puts it in some kind of infinite loop and I need to find out why.
That being said there's a simple solution now. You can disable the fallback DNS feature by doing this:
ha dns options --fallback=false

This is better then leaving the feature enabled but blocked.
But yea at some point I will figure out why this happens and put a stop to it.

I'm not familiar with the code, but the overall control flow looks like it could become unintentionally re-entrant if the chain of handlers was set up incorrectly somehow: https://github.com/home-assistant/plugin-dns/blob/master/plugins/fallback/fallback.go#L62

Yea I was thinking that too. But what's weird is its set to fallback on all of these codes:
https://github.com/home-assistant/plugin-dns/blob/master/rootfs/usr/share/tempio/corefile#L24
So I would think if it was that it would also have this issue if you requested some non-existent name and got an NXDOMAIN response. I can't figure out what would be special about SERVFAIL.
But yea, I do think it might be that even if I can't explain it right now.

Hmm.  I suppose the loop could be further upstream in whatever code is triggering resolution in the first place.  Ahh well, good luck!

Just run into a wired state which my firewalls disk was full and not working anymore.
My FW drops/blocks DNS/DoT to the Internet as the local one take care of this completly.
I saw that the log file for 2 days grow to more than 50GB!
While analysing the log file I can see that the a absolutli HUGE amount (more than 20 request/second) comres from HA which is trying to reach <IP_ADDRESS> and <IP_ADDRESS> with Port 853.
Can we please have an option to disable this behavior as in my case it is totally unneeded and unwanted.

There is already a way to disable the feature.  See @mdegat01's comment above: https://github.com/home-assistant/plugin-dns/issues/90#issuecomment-1119139222
This issue is about fixing the looping bug.

Hi @CentralCommand, I've been able to root cause the runaway DNS queries as a result of what I call infinite looping configuration in /etc/corefile. The lines of configuration that are contributing to this issue are loop, fallback REFUSED,NXDOMAIN, and max_fails 0. Here's what happens:

CoreDNS starts and executes the loop plugin, which sends a query for <random number>.<random number>.zone that gets forwarded externally and the Root Servers respond with NXDOMAIN.

CoreDNS triggers the fallback plugin due to the NXDOMAIN response.

CoreDNS now sends all queries, including health_check (NS IN .), to Cloudflare over TLS.

User's firewall blocks DNS over TLS (TCP 853).

CoreDNS triggers the fallback plugin due to the REFUSED response.

Since max_fails 0 is set, CoreDNS assumes Cloudflare is always healthy.

CoreDNS is now in an infinite loop continuously sending and retrying its health_check query. :nauseated_face:

Initial thoughts for this PR to fix this are:

HINFO query from the loop plugin should not trigger fallback.
Allow users to specify whether or not to use DNS or TLS when configuring the plugin-dns container.
Don't assume Cloudflare will always be available.
[Unrelated]: Remove policy sequential so not to overload a single user's DNS server.

Although, before reworking this configuration, is the narrative on why plugin-dns exists in the first place is to provide continuous access to well-functioning and available DNS servers?

@mdegat01, as mentioned here  I've been able to root cause the runaway DNS queries as a result of what I call infinite looping configuration

Thanks for that awesome troubleshooting! One strange thing. I NAT transfer DNS over TLS to my firewall vs just plain blocking it. I thought a misconfiguration triggered this, but no - the redirect is working. I'm able to intercept <IP_ADDRESS>:853 traffic in other applications. So, it should be getting a valid response. I'm assuming HA is (technically correctly) rejecting the cert from my firewall.
There's no "problem" with HA using DNS over TLS as the default IMO but it should accept the network authoritative server.

I discovered the same issue.
As I force every network client to use my personal DNS Server (Pi-hole running unbound), and blocking every other DNS port in my router, I noticed that plugin-dns spams my network with cloudflare requests (resulting in high CPU/network usage).
As for now, I'm using https://github.com/bentasker/HomeAssistantAddons/tree/master/core-dns-override to manually override DNS settings.

From @d0nni3q84:

Although, before reworking this configuration, is the narrative on why plugin-dns exists in the first place is to provide continuous access to well-functioning and available DNS servers?

My understanding is that the plugin handles two things. It handles making sure container names can be resolved by all containers, even those that use host networking, and via the fallback it works around a nasty issue where locally hosted DNS software interacts very poorly with alpine containers (because MUSL acts differently from GLIBC), which heavily impacts home assistant, where basically all the containers are Alpine Based.
Resolving containers by name in all containers
The idea here is to make it possible for the Home Assistant Core container to run in the host networking namespace while still being able to resolve containers by name. This also same applies for add-ons that opt into using the host networking namespace.
You see normally docker containers can refer to each other by name thanks to Docker running a stub DNS server on <IP_ADDRESS> which handles the names that docker controls and resolves others using the host's resolver. Unfortunately, this is not available to containers that run in the host's network namespace.
This is true even if the docker network is made available to the host, which makes it possible to access those containers by IP address.
By running a separate DNS server whose IP address is known to the supervisor, all the user facing containers can be configured to use it as their DNS server, and they can all resolve these names.
The nasty issue requiring fallback
Some software that users may be using locally to handle DNS handles missing AAAA records incorrectly by returning NXDOMAIN instead of NOERROR with an empty result set.
The problem is that MUSL assumes that the DNS servers are standards conformant, so when when looking for either A or AAAA records at one (the "happpy eyeballs" algorithm), if the AAAA record comes back as NXDOMAIN, it will not wait for the A record to come back, but immediately report that the name cannot be resolved. If the DNS servers used are strictly conformant this would never be an issue, but it is with the local software some uses have.
The end result is that some sites that are IPV4 only could randomly become inaccessible to Home Assistant, and for no obvious reason.  Having this fallback system allows avoiding this, by asking Cloudflare if an NXDOMAIN came back, and returning a proper NOERROR instead.  And if that is being done, it might as well try to handle other errors too, as some of them may be due to such software not understanding the results it got back from a recursive server.
The above is basically just a summary of what was described in this forum post.
Supervisor is now able to detect if this condition is happening, so they now allow you to disable the fallback. However, if you use that setting when you have a broken local DNS server you system will be marked as unsupported.
Originally the fallback was also available for use if all other servers got marked unhealthy, but for unclear reasons, the health checks were not working to recover the other DNS servers for some users, with undesirable consequences.

As for the rest of your analysis, there are some problems with it, although you do point out a key issue. In a follow-up post, I'll go over what is happening.

What is causing all these requests?
@d0nni3q84's analysis is close, but a little but off. (Like claiming that all requests were getting forwarded to the fallback). Rather than reply point by point I just give a correct explanation below.
Lets look at the tail end, first. What happens to an incoming request to port 5553 while Cloudflare is blocked?
Because max_fails 0 is set, it will spend 5 seconds repeatedly trying to contact both cloudflare servers in a loop. This would not happen if max_fails were not zero. After the 5 seconds are up it will return SERVFAIL.
So if the main chain's forward plugin ever returns an NXDOMAIN response, we will get a 5 second burst of attempted trafic to Cloudflare if it is blocked? No. It is much worse than that!
It turns out that that the forward plugin being used internally by the fallback plugin will only wait to read a response for two seconds, before declaring the request failed, and moving onto the next server, which is actually the same one again if not yet marked as failed. This will result in more than one overlaping 5 second mini DOS attack on the router firewall. But this is not the worst part.
No the worst part is that taking longer than two seconds means the forward plugin used by the fallback plugin thinks :5553 has failed, and begins sending health check messages every half second (since that is the default)! Each of these health check messages will result in 5 seconds of spamming the router firewall.
You were correct that the loop plugin would ensure that an NXDOMAIN lookup happens early in the process. It is also not great since the zone is ".", so the lookup is being sent to "<random_number>.<random_number>.", which obviously your preferred recursive resolver server won't have cached, and will need to ask the root DNS servers for every time, which means this plugin is putting unnecessary load on the root DNS servers. (Footnote 1)
So what is up with the 5 second thing?
The only backend plugin in the fallback chain is the forward plugin.  The forward plugin loops though a list of configured servers (the list may be permuted by the policy setting). It will try to contact each server in the list, skipping any that are marked as unhealthy. If it gets anything it can parse as a DNS response, it will return that, and the process ends. It continues to try more servers until it has reached a hardcoded 5 second timeout.
If it reaches the end of the list and there is still time on the 5 second timeout, it starts again at the beginning of the list. Unfortunately, because both Cloudflare servers are unreachable in this scenario, it will alternate back and forth between the two try over and over again until the 5 seconds have elapsed. If being blocked by a firewall, and thus getting an RST back, this could happen in a quite rapid fire way.
There is however a special case. If all of the servers in the list are unhealthy, then it will pick one at random and try it. If that one fails, it is done, and breaks out of the loop without needing to wait for the 5 seconds to be up. Unfortunately if max_fails is set to zero, the servers won't end up getting set to unhealthy.
This means that trying to avoid health check probes to Cloudflare by setting max_fails 0 actually made this meaningfully worse, as before then, the servers would get marked as down, and would only get one randomly selected as an attempt for each.
So what should be done?
The fallback chain should be configured with a template plugin to reply to the health check without forwarding. Even just returning REFUSED or SERVFAIL would help. This would prevent from piling up and compounding problems if Cloudflare is blocked. The fact that this would prevent :5553 from returning the nameservers for the root zone is not an issue, since :5553 is only used when an error has already occurred, and the other resolvers should have been able to handle that request without error.  This alone would prevent the endless CPU usage, but would not prevent the 5 second mini DOS attacks on user's firewalls.  The next suggestion would prevent that. (It should make this one unneeded too in theory, but defense in depth is probably better here).
The fallback chain really needs to re-enable health checks. If they work the way they are supposed to, they won't start getting sent until the first failed request to Cloudflare, and they stop instantly after after getting back something that looks like a DNS response. (Footnote 2) One request every 5 minutes indefinitely is far better than what max_fails 0 is doing with sending rapid fire requests.
More responses to @d0nni3q84

Initial thoughts for this PR to fix this are:

HINFO query from the loop plugin should not trigger fallback.

This would conflict with your next point. It would arguable be okay to do this if DNS over TLS or DNS over HTTPS is enforced (as it must be anyway, see next response).
Ignoring that though with normal TCP or UDP, if the router is potentially intercepting requests being made to Cloudflare, those requests could form a loop, and should be detected. (Footnote 3) Excluding that request only makes sense .

Allow users to specify whether or not to use DNS or TLS when configuring the plugin-dns container.

The whole point of the fallback is that the local router cannot intercept it and do the wrong thing, so it really needs to remain using TLS (or DNS over HTTPS).

Don't assume Cloudflare will always be available.

Yeah, this was a real problem.

[Unrelated]: Remove policy sequential so not to overload a single user's DNS server.

That would cause issues with letting people specify preferred DNS servers via ha dns options --servers, since those get prepended to the servers found via DHCP. Sequential allows for preferentially using the user specified ones, but falling back to the DHCP provided ones if the user specified ones don't respond at all is preferable.

Footnotes

Footnote 1 (click to expand)
 This could be addressed by having the first zone listed for the main plugin pipeline be a zone that definitely does not exist, like `looptest.home-assistant.io.`, or `invalid.`. These would yield lookups for `..looptest.home-assistant.io` or ``..invalid`. The important thing is that there be an NXDOMAIN that the recursive resolver can cache so it does not need to ask the root servers every time. The loop plugin uses the first zone defined for the plugin chain, so you would simply replace `.:53 {...}` with `invalid .:53 {...}` to do that. (The syntax allows multiple whitespace  separated zones before the port number) Obviously being nice to the root servers is not required.

Footnote 2 (click to expand)
Of course perhaps the health checks are buggy. I'm still not sure that the cause is known for why health checks did not restore local DNS as preferred back when the fallback was in the main forward plugin.

Footnote 3 (click to expand)
You could have a scenario like follows: Router intercepts these requests and is configured to forward those them to a PiHole-like system running in a Home Assistant addon. That addon uses the CoreDNS server as its upstream. The loop addon would detect the obvious loop, so the user sets DNS servers with `ha dns options --server` to some upstream DNS that is whitelisted from interception by the router. Unfortunately there is still a loop that would happen for any non-existent domains queried, which would hit the fallback case, try to forward to Cloudflare, get intercepted and forwarded to the Pi-Hole like software, which asks CoreDNS again...  The loop plugin would detect this, but it can only do so if fallback does not block its query.

This really needs fixed. I'm brand new to setting up Home Assistant. I setup the latest VM (VMware ESXi/vSphere) and my network uses DHCP, and assigns a valid and working DNS server. And I'm getting blocked firewall logs constantly for access to <IP_ADDRESS>:853 and <IP_ADDRESS>:853. As far as I am concerned, there is absolutely zero reason for this VM to ever reach out to these IPs.

Running the ha dns options --fallback=false did fix the problem for me, though. But I should not need to run that on a new setup, with a valid DNS setup.
BTW, I did not read any of the reasoning for why this happens or what it is trying to do-- As a server admin, I would never expect such an aggressive query against blocked DNS servers, especially when there is a valid DNS configured for the host. Having to run the google search and parse through forum posts and several closed github issues before any mention of a quick workaround is not a good experience.
I do appreciate any effort put into resolving this issue, though. :-)

I setup the latest VM (VMware ESXi/vSphere) and my network uses DHCP, and assigns a valid and working DNS server

For anybody reproducing this, I think this is the key. I doubt most users would really notice this running on a raspberry Pi or similar low-power hardware. I put it in a VM with a virtual 10 GBps network connection to my firewall/router in the same VM. Both HA and the firewall were very low latency and snappy to respond. Under that situation, Home Assistant is aggressive enough to nearly saturate the 10 Gb connection with DNS requests and commensurately burn some serious CPU power in both HA and the firewall 24/7.

Ok, I just hit the same issue again.  I set up a fresh instance of Home Assistant and several days later I finally notice that it's been spamming my network with gigabytes worth of pings to <IP_ADDRESS> -- out over StarLink!
Please can we do something to fix this?  At the very least try not to flood the network.

I can confirm that running ha dns options --fallback=false does indeed not fix the issue. This is happening since I set the DNS server distributed via DHCP to my HA instance running AdGuard Home. HA is configured to use a static IP and another local DNS server (not itself via AdGuard Home).

Agree with @kode54 that the proper way would be to use the Canary domains.
First of all because that is 'best practice', secondly because it avoids confusion for those not familiar with the command to disable the fallback (which could happen if you're a first time user, let's be honest) and thirdly because there is no point in running coredns whatsoever except in case of issues (e.g. when  the canary domains don't work).
This should be much more 'opt-in' or 'in case of issues' rather than the partial 'opt-out' today.

Im confused now.
I set ha dns options --fallback=false and set a static DNS server in my IPv4 config, yet with a capture I see not 1 DNS request sent to my DNS server, instead sent to <IP_ADDRESS> and my gateway?
None of these addresses are even handed out as DNS from DHCP.
How are you guys defining your static DNS servers?

I have my Pi-Hole address handed out from DHCP as well, and the router also acts as a firewall and catches naughty DNS requests to other addresses and redirects them.
Very much stunned that HA is not respecting DNS addresses - and ANOTHER reason to make this coredns stuff opt-in.

Just did a first installation of Home Assistant OS on rpi3 just to give it a try and test some of zigbee-enabled hardware. My pi is burning with 70+ degrees with coredns in the top for CPU time and I'm seeing this in logs:
➜  log docker logs hassio_dns 2>/dev/null | tail -30
[INFO] <IP_ADDRESS>:38539 - 28343 "NS IN . udp 17 false 512" NOERROR - 0 5.007165535s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:58223 - 23638 "NS IN . udp 17 false 512" NOERROR - 0 5.001772123s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:43302 - 63044 "NS IN . udp 17 false 512" NOERROR - 0 5.001283552s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:51113 - 35362 "NS IN . udp 17 false 512" NOERROR - 0 5.003702062s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:38262 - 56755 "NS IN . udp 17 false 512" NOERROR - 0 5.002333335s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:33174 - 9323 "NS IN . udp 17 false 512" NOERROR - 0 5.003397732s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:33094 - 32776 "NS IN . udp 17 false 512" NOERROR - 0 5.010711032s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:44416 - 48101 "NS IN . udp 17 false 512" NOERROR - 0 5.002663142s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:35474 - 16537 "NS IN . udp 17 false 512" NOERROR - 0 5.005863059s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:50759 - 43058 "NS IN . udp 17 false 512" NOERROR - 0 5.000412148s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:33519 - 45040 "NS IN . udp 17 false 512" NOERROR - 0 5.004900032s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:39739 - 59567 "NS IN . udp 17 false 512" NOERROR - 0 5.005401149s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:39157 - 50430 "NS IN . udp 17 false 512" NOERROR - 0 5.00624565s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:48527 - 47596 "NS IN . udp 17 false 512" NOERROR - 0 5.003011352s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused
[INFO] <IP_ADDRESS>:60637 - 59663 "NS IN . udp 17 false 512" NOERROR - 0 5.001902262s
[ERROR] plugin/errors: 2 . NS: dial tcp <IP_ADDRESS>:853: connect: connection refused

So my guess is that I'm hitting one of the issues highlighted in this report. I do have an openwrt-powered router with DoH enabled. Any pointers on how to normalize this are greatly appreciated.
Version info: Home Assistant 2023.10.3 Supervisor 2023.10.0 Operating System 11.0 Frontend 20231005.0 - latest

While looking into CoreDNS in general today, I've reproduced this problem on my end today. Just adding a reject firewall rule for port 853 TCP let's it easily reproduce.
@KevinCathcart your analyzes seems spot on. Some points to add:

This means that trying to avoid health check probes to Cloudflare by setting max_fails 0 actually made this meaningfully worse, as before then, the servers would get marked as down, and would only get one randomly selected as an attempt for each.

While not explicitly stated, I guess one of the attempts in https://github.com/home-assistant/plugin-dns/pull/82 was also to avoid health checks when the primary DNS server is working. Essentially, make sure we don't contact Cloudflare if not necessary. But currently, the loop plug-in will still cause at least one resolve attempt. And for the reasons you've outlined, this is problematic when access to Cloudflare using DNS over TLS is blocked.
I've tested your suggestion of adding a template plug-in to avoid health checks and re-enable health checks on Cloudflare (using max_fails). This works exactly as you've suggested.
It does lead to 30s long storm whenever the loop plug-in is probing on first startup still (30s comes from the loop plug-in default behavior: "will try to send the query for up to 30 seconds"). I think it is safe to assume that Cloudflare won't lead to a DNS loop, so I went with another template plug-in to handle the loop request. This also avoids the (additional) request to the root servers through Cloudflare, and any request to Cloudflare when the primary DNS is working (so also in the case when Cloudflare using DNS over TLS is reachable).


Remove CherryPy version requirement

cheroot v8.6.0 has been released, used in CherryPy v9.0 and above, which has the high idle CPU load resolved: https://github.com/cherrypy/cheroot/pull/401
Since this was the reason for using a CherryPy<9.0 version dependency, this is hereby removed.

Did some quick performance test:
CherryPy v8.9.1
2022-01-04 14:49:51 root@VM-Bookworm:~# time for i in {1..100}; do curl -sI http://<IP_ADDRESS>:8085 -o /dev/null; done

real    0m4.400s
user    0m1.038s
sys     0m0.702s
2022-01-04 14:50:08 root@VM-Bookworm:~# time for i in {1..100}; do curl -sI http://<IP_ADDRESS>:8085 -o /dev/null; done

real    0m4.441s
user    0m1.136s
sys     0m0.616s

And ~0.14 seconds idle CPU load per minute
CherryPy v18.6.1 with cheroot v8.6.0
2022-01-04 14:56:39 root@VM-Bookworm:~# time for i in {1..100}; do curl -sI http://<IP_ADDRESS>:8085 -o /dev/null; done

real    0m4.522s
user    0m1.105s
sys     0m0.750s
2022-01-04 14:56:55 root@VM-Bookworm:~# time for i in {1..100}; do curl -sI http://<IP_ADDRESS>:8085 -o /dev/null; done

real    0m4.450s
user    0m1.096s
sys     0m0.668s

Probably not a great benchmark, just the quickest that came to my mind 😄
And ~0.35 seconds idle CPU time per minute. Still worse but much better than before (with cheroot v8.5.2 it's ~2.25 seconds CPU time per minute, more than 6x higher), sufficiently small in absolute numbers I'd say to list the CherryPy version freeze.

Merging this. Further performance tests have been done and it should be totally worth it.


convert cti to inp files (not an error)

Hi there,
I use your pymras to convert cti file to inp file, then input the inp file in chemkin to do simulation. when doing pre-processing in Chemkin, I got an error about Arrhenius parameter,
this is the converted inp file, need to notice, there is no space on the red marked and pointed reaction,

and this is the Chemkin error log:

so I don't know whether it is my fault during generating cti, or you may or could give more space between reaction and Arrhenius parameters in your converting cti to inp script.

Hi @shenghuiqin, can you share both the original CTI file and the generated Chemkin file?

reduced_338.zip
eg, line 4688 on inp file, you can see the connected reaction species and A-factor
thank you for your time.

Hi @kyleniemeyer, I believe that my version of soln2ck will not have this problem. I did quite a bit of reformatting the output file to eliminate such issues and make it more legible. I cannot remember if I altered the inputs into the main write function. There is also the additional functionality of adding notes back into the output chemkin-style mechanism.
My soln2ck


Fix the problem that startDate and endDate may not be converted.

I found sometimes 'endDate' and 'startDate' are not converted even though the component should also handle the string formatted date.
I fixed this problem and added a test case to confirm the problem was resolved.

this is great, thanks for the PR @kenkoooo!
published as v1.6.3


Case insensitive comparisons for no-target-blank

<a href="foo" target="_BLANK"> should also trigger no-target-blank.
Now it does.

Do browsers natively treat them as case-insensitive (and does react normalize the case)? If not, they should not be targeted by this rule.

Looks like I was overzealous with the HTML attribute name. Those seem to be case sensitive. But target="_BLANK" definitely works: http://codepen.io/dmnd/pen/LRPmZE?editors=1010

Amended to not include all-caps html attributes.

Thanks. Can you rebase your branch ? Then I think we'll be good to go.

Rebased.

Perfect, thanks!


how can you search prime numbers with in a certain range like

searching prime numbers between 100000<range<1000000 how to make this ranged  search primes with in a certain range

Range based search using primesieve command-line program: primesieve 100000+1 1000000-1 --print
Range based search using C API: https://github.com/kimwalisch/primesieve/blob/master/doc/C_API.md#primesieve_jump_to-since-primesieve-110
Range based search using C++ API: https://github.com/kimwalisch/primesieve/blob/master/doc/CPP_API.md#primesieveiteratorjump_to-since-primesieve-110

Please note that for simple use cases you can often write a bash script using the primesieve command-line program. However for more complex use cases the C/C++ API is better suited. If you don't know C or C++ you can use primesieve bindings for other programming languages such as e.g.: https://github.com/shlomif/primesieve-python.


3D Tiles Inspector freeze frame broken for some tilesets

It works for the official demo with NYC, but other tilesets I tried cause it to crash.
The crashing tileset has a texture in it, maybe that's the cause?
CC @lilleyse

This seems to happen with most tilesets now.  Including the stock Sandcastle demo.
Anyone interested in taking a look at this?

Hey I can't reproduce this.

// San Miguel model created by Guillermo M. Leal Llaguno. Cleaned up and hosted by Morgan McGuire: http://graphics.cs.williams.edu/data/meshes.xml
var viewer = new Cesium.Viewer('cesiumContainer');

viewer.extend(Cesium.viewerCesium3DTilesInspectorMixin);
var inspectorViewModel = viewer.cesium3DTilesInspector.viewModel;

var tileset = new Cesium.Cesium3DTileset({
    url: Cesium.IonResource.fromAssetId(19365)
});
viewer.scene.primitives.add(tileset);

var initialPosition = new Cesium.Cartesian3(-1111583.3721328347, -5855888.151574568, 2262561.444696748);
var initialOrientation = new Cesium.HeadingPitchRoll.fromDegrees(100.0, -15.0, 0.0);
viewer.scene.camera.setView({
    destination: initialPosition,
    orientation: initialOrientation,
    endTransform: Cesium.Matrix4.IDENTITY
});

Use the above code
With the default view, while it's loading hit freeze frame
Zoom out slightly and mouse around a little bit and it crashes. (you may have to rotate the camera  a little as well).

Sorry I didn't leave clear repro instructions originally but I think it happened more frequently then.

It's much easier to reproduce it in this Sandcastle.


Is there a way to get cors when using DD? it makes SPA's a pain when you can't use DD because it just throws a cors error on the FE

This started to annoy me too, but I found a little trick.
First I wanted to use dd, I did not want to make a different function since I was just so used to dd() that it was still going to trip me up, trouble is dd is defined in the autoloader and not the bootstrap process, so you basically need to put it in public/index.php before the autoloader in order to override the regular dd function (since the regular function looks for the function already existing, it just will not define it).
This is how I did it.
if (! function_exists('dd')) {
    /**
     * Dump the passed variables and end the script.
     *
     * @param  mixed  $args
     * @return void
     */
    function dd(...$args)
    {
        header('Access-Control-Allow-Origin: *');
        header('Access-Control-Allow-Methods: *');
        header('Access-Control-Allow-Headers: *');
        http_response_code(500);

        foreach ($args as $x) {
            (new Illuminate\Support\Debug\Dumper)->dump($x);
        }

        die(1);
    }
}

Put this below define('LARAVEL_START', microtime(true)); and you will be able to use dd normally with cors.

Nice!

Just in case anyone else finds this and is using Laravel >= 5.7.x, you'll need to replace Illuminate\Support\Debug\Dumper with Symfony\Component\VarDumper\VarDumper

If you are doing requests with credentials replace:
header('Access-Control-Allow-Origin: *');

with
$origin = $_SERVER['HTTP_ORIGIN'] ?? '*';
header("Access-Control-Allow-Origin: $origin");
header('Access-Control-Allow-Credentials: true');

If anyone keeps coming back to this answer to temporarily paste the above dd method into their public/index.php file and is tired of doing so, you can tell git to locally ignore all changes to that file ( since it shouldn't really ever change for a Laravel project by doing git update-index --skip-worktree public/index.php
Now you can have your dd locally and it won't get committed to your branches.
In the event you do want to make a change to that file, simply run git update-index --no-skip-worktree public/index.php and commit away!

I'm closing this due to inactivity. If this issue is still relevant on the latest version (2.0.1 at the moment), please open a new issue with detailed information.

This started to annoy me too, but I found a little trick.
First I wanted to use dd, I did not want to make a different function since I was just so used to dd() that it was still going to trip me up, trouble is dd is defined in the autoloader and not the bootstrap process, so you basically need to put it in public/index.php before the autoloader in order to override the regular dd function (since the regular function looks for the function already existing, it just will not define it).
This is how I did it.
if (! function_exists('dd')) {
    /**
     * Dump the passed variables and end the script.
     *
     * @param  mixed  $args
     * @return void
     */
    function dd(...$args)
    {
        header('Access-Control-Allow-Origin: *');
        header('Access-Control-Allow-Methods: *');
        header('Access-Control-Allow-Headers: *');
        http_response_code(500);

        foreach ($args as $x) {
            (new Illuminate\Support\Debug\Dumper)->dump($x);
        }

        die(1);
    }
}

Put this below define('LARAVEL_START', microtime(true)); and you will be able to use dd normally with cors.

How did you do it, am using laravel 8+ but cant make the dd work on api.


Draft: utils.py: Remove the (?ms) switch at the start of _glob2re

See original merge request on GitLab
In GitLab by [Gitlab user @BenjaminSchubert] on Aug 28, 2020, 15:00
The ?ms switch stands for multiline output.
This switch is used nowhere in our codebase.
It also is problematic in multiple places where its use is deprecated since python 3.7,
as now they need to be at the start of an expression, which is not the
case in all the calls to it that we are doing.
If someone wants the same functionality, they should add it explictely
after calling _glob2re
TODO
Still need to add a test to see how BuildStream behaves with split files that have such characters in them

In GitLab by [Gitlab user @BenjaminSchubert] on Aug 28, 2020, 15:01
changed the description

In GitLab by [Gitlab user @BenjaminSchubert] on Aug 28, 2020, 15:01
changed the description

In GitLab by [Gitlab user @BenjaminSchubert] on Aug 29, 2020, 11:58
marked as a Work In Progress

In GitLab by [Gitlab user @BenjaminSchubert] on Aug 29, 2020, 11:58
marked as a Work In Progress

In GitLab by [Gitlab user @BenjaminSchubert] on Aug 29, 2020, 11:58
changed the description

In GitLab by [Gitlab user @BenjaminSchubert] on Aug 29, 2020, 11:58
changed the description


Update README.md

Fixed build status badge url.

Did you update the tests?

[ ] yes
[ ] no
[x] n/a

Did you update the docs?

[x] yes
[ ] no
[ ] n/a

What do you think about renaming it from .yaml to .yml to match the other repos?

Haha :) I am fine with that as well. Actually, I was thinking about the same but saw other repositories with the .yaml extension.


Add deduplication functionality

Introduce generate:deduplicate script
This script removes duplicate entries that would never "win" the race the matcher runs to find the first match in a given language. For example, the word "the" will match the en/wikipedia file first because it's the 1st entry. It's the 2nd entry in en/commonWords, but that ends up just taking space and increasing the commonWords list unnecessarily.
This script will load all the words for each language, filter out the duplicate words that have a worse "rank", and then re-write the files. It runs in ~3 seconds on my machine. :)

Results of running du -k (disk usage in kilobytes) for packages before / after the deduplication script below:
common: 496 -> 496 (±0%)
de: 1060 -> 932 (-12.1%)
fr: 868 -> 672 (-22.5%)
en: 1496 -> 1336 (-10.7%)
nl-be: 604 -> 480 (-20.5%)
total: 4616 -> 4008 (-13.2%)

This is an awesome idea to reduce language bundle size! 👍


Defined new revisions for golang dep: github.com/containernetworking/cni

Ran component through the scanners since I noticed it had not been harvested yet. PR matches curation site. Closing.


Change prologue details to prepare for fsdp as a transform

support tracing a model on meta device (to enable tracing before materializing / sharding)
use model.get_parameter / model.get_buffer to acquire parameters and buffers to more easily divert these.

Thank you @IvanYashchuk @crcrpar . Let's think about the meta bits separately. I'm happy to switch to whatever works best for fsdp.


Update and Reorganize Documentation

Documentation exists for this project as a single solution.  The R2 infrastructure will have to be maintained in the project but needs to be enhanced to support R3 with an AKS based micro-service solution.  Documentation structures have to be moved to enable concise documentation for each architecture.
Acceptance Criteria

Root Level Readme should explain the project and have links to different solution documentation.
Solution documentation should be structured
Architecture Diagrams should exist
Hyperlinks for design information of the project should be valid.

This has been completed with PR #34.


Get rid of coredns sidecar for nsc to simplify NSM usage in k8s

Context
https://github.com/networkservicemesh/deployments-k8s/tree/release/v1.4.0/examples/features/dns
Description
Currently we're using the coredns side car as a proxy dns server that fanounts dns queries to different dns worlds (in particular k8s dns)
We're using WH to inject this sidecar for nsm clients
https://github.com/networkservicemesh/cmd-admission-webhook-k8s/blob/main/main.go#L226-L242
Problems

workload 1/1 is  turning  to 3/3 when we add nsm annotation in k8s (we could reduce it to 2/2 where 1/2 - workload, 2/2  nsc)
we need to add shared volume in pod https://github.com/networkservicemesh/cmd-admission-webhook-k8s/blob/main/main.go#L163-L171
coredns costs resouces that is critical for horizontal scaling.
coredns reload plugin is too slow (it tooks ~2s to reload configuraiton that could be super critical for complex production cases)
we need to deploy/manage fanout plugin

Solution

Remove coredns usage
Run simple dns server (based on coredns main lib https://github.com/miekg/dns) as a goroutine

Note: if we'll need to get a special Coredns functionally we could simply imprort the plugin and adapt it for our dns server. https://github.com/networkservicemesh/sdk/blob/main/pkg/tools/dnsutils/adapt/handler.go#L17
Potential dns chain elements could be
func main() {
...
    var dnsServerHandler = next.NewDNSHandler(
          resolveconf.NewDNSHandler(),
          dnsconfigs.NewDNSHandler(),
          searches.NewDNSHandler(),
          noloop.NewDNSHandler(),
          norecursion.NewDNSHandler(),
          cache.NewDNSHandler(...),
          fanout.NewDNSHandler(....),
     )
  go dnsutils.ListenAndServe(dnsServerHandler, ...)
...
}

Some of chain elements alerady done see at https://github.com/networkservicemesh/sdk/tree/main/pkg/tools/dnsutils

/cc @edwarnicke , @fkautz

TODO
SDK

Rework fanout   dns chain element to force work with context isteadof function.
Rework dnscontext.NetworkServiceClient => create resolvconf dns chain element. That simply edits resolv.conf file by analog with  dnscontext.NetworkServiceClient
Rework dnscontext.NetworkServiceClient => create new dnsconfigs dns chain element via shared map. That adds into context fanout list and searches.
create new searches dns chain element that adds for question search sub domains.
create new cache dns chain element that stores answers for TTL period.

cmd-nsc, cmd-nsc-init

Add new goroutine dns server

cmd-admission-webhook

Remove everything that related to Coredns.

Defenition of done
All subtasks are completed https://github.com/networkservicemesh/deployments-k8s/tree/main/examples/features/dns is working without changes.

This one is resolved!
@NikitaSkrynnik Thanks!


comment

this is a remarkable tool/service/capability.
it shows us how to do something we've struggled with for a while.
thank you for the building blocks.

Thank you @jaygray0919 for your kind words. I'm very glad you find shaperone useful. I hope you would actually use it in your projects. Let me know if I can help in any way
And if you're looking for inspiration, I will be more than happy to share my experience :)


Android getAllContacts not working

red screen
attempt to invoke interface method boolean android.database.Cursor.moveToNext() on a null object reference

Occured at getAll ... ContactsManager.java:38

"Read contacts... 2 mins ago... Allowed 9843 times"
Something must really be off!

yikes x.x
I have no specific timeline for addressing this, but I will post here when and if I am jumping in

fwiw, if anyone wants to look into this, I think the better approach here is to adapt code from an existing package like cordova contacts plugin: https://github.com/apache/cordova-plugin-contacts/tree/master/src/android

@rt2zz I tried patching it, but getAll doesn't work.  Not sure if this is due to some change in Lollipop.  I'm not sure really.  I would gladly pay a $50-100+ bounty if someone fixed this.

@elliottsj can you help?

@drFabio per our chat!

Can anyone help here? @rt2zz please re-open  /cc @morenoh149

$250 bug bounty

I'll look into it

It's definitely working. Try performing all the steps in the setup guide.

No it's not
On Dec 9, 2015 8:23 PM, "Harry Moreno"<EMAIL_ADDRESS>wrote:

It's definitely working. Try performing all the steps in the setup guide.
[image: screen shot 2015-12-09 at 8 22 22 pm]
https://cloud.githubusercontent.com/assets/587438/11704096/abbc09c4-9eb2-11e5-9696-7539166925c3.png
—
Reply to this email directly or view it on GitHub
https://github.com/rt2zz/react-native-contacts/issues/10#issuecomment-163456799
.

Try it with 300 contacts, try to break it like I did and got those cursor
errors
On Dec 9, 2015 8:24 PM<EMAIL_ADDRESS>wrote:

No it's not
On Dec 9, 2015 8:23 PM, "Harry Moreno"<EMAIL_ADDRESS>wrote:

It's definitely working. Try performing all the steps in the setup guide.
[image: screen shot 2015-12-09 at 8 22 22 pm]
https://cloud.githubusercontent.com/assets/587438/11704096/abbc09c4-9eb2-11e5-9696-7539166925c3.png
—
Reply to this email directly or view it on GitHub
https://github.com/rt2zz/react-native-contacts/issues/10#issuecomment-163456799
.

how do I load >300 contacts into the simulator?

@morenoh149 see https://stackoverflow.com/questions/4835652/how-to-put-contacts-on-the-android-emulator, I think you use Virtual SDCard.

@morenoh149 you can create contacts with the API here https://stackoverflow.com/questions/4744187/how-to-add-new-contacts-in-android

see https://github.com/apache/cordova-plugin-contacts/tree/master/src/android

@morenoh149 also see this code here already done https://github.com/apache/cordova-plugin-contacts/blob/master/src/android/ContactAccessorSdk5.java#L329-L336

It'd be really great if you could upload a sample .vcf file with >300 contacts to gist.github.com for me @niftylettuce

Might be more productive to implement addContact to android and see what happens when we add >300 contacts

Yeah - maybe try that and then iterate it over 300 with fake names or
something?
On Thu, Dec 10, 2015 at 12:43 PM, Harry Moreno<EMAIL_ADDRESS>wrote:

Might be more productive to implement addContact to android and see what
happens when we add >300 contacts
—
Reply to this email directly or view it on GitHub
https://github.com/rt2zz/react-native-contacts/issues/10#issuecomment-163699472
.

https://gist.github.com/niftylettuce/6202138b9ab82f9f8368

I found one with two http://www.phpclasses.org/browse/file/5543.html

@morenoh149 Anything I can do to help?  Do you need some of the bug bounty up front?


Add support for posting comments on commits as an alternative to pull requests

Add support for supplying sonar.stash.commit.id as an alternative to sonar.stash.pullrequest.id in order to get comments on a commit instead of a pull request if desired by the user.
This commit also tidies up some line endings where there were some unix line endings mixed in with predominately DOS files.

Hello John,
Thanks for the contribution. We will review this alternative asap.
Regards,

Hi @johnmdilley,
I adopted, rebased etc. this PR and created a new one based on it in #65, therefore I'll close this PR.


V0.5rc siqi

Coverage decreased (-0.4%) to 81.534% when pulling 352aba6e3aef52fe9dc396d42421a3528ba6e9b0 on V0.5rc-siqi into e73621e967baebc1fb7193f86806d32aa3a1b903 on V0.5rc.


Add new helper specs

This replaces some of the old tests with new, cleaner specs.
Hopefully, once everything is converted into nice specs, the helper context can become part of the public API. That’s not the goal of this PR, however.

This does not replace all helper tests, but the existing ones are too high-level or complicated to rewrite in a nice way. Might do this later.


iOS 15.2 Apps built on Xcode 13.2 app is terminating on TwilioVoiceSDK.handleNotification

I am facing issue in iOS 15.2 with iOS Voice SDK 6.3.0
Call works perfectly fine when app is in active state but when app goes in BackGround or app is killed then call didn't work. Here are the logs for help.

*** First throw call stack:
(0x184419d3c 0x19b77e6a8 0x185bea99c 0x1ba01003c 0x110395f94 0x1103a5638 0x1ba00f35c 0x110394718 0x110395f94 0x1103a4758 0x1843d5888 0x184393188 0x1843a5e1c 0x1a47009a0 0x186bd9600 0x18696eb08 0x104081228 0x10deb4250)
libc++abi: terminating with uncaught exception of type NSException

Hi @warisappify
Please follow the migration guide and make sure your application reports to CallKit immediately upon receiving the VoIP push notification. You can also check out how the quickstart does this.
Remember:

Initialize your PushKit instance in AppDelegate to ensure it gets created at app launch
Report to CallKit immediately upon receiving the VoIP push notification

Ooohh man i think you didn't read the description and close the issue.
If i didn't follow the migration and other steps you mention then why call received in active state?

The app is getting terminated because the app fails to follow the policy of PushKit + CallKit, as indicated in the debugging log. This has been reported multiple times and this happens especially when the app is not active. Make sure you follow the implementation best practice in our quickstart and apply necessary changes to your application.

I did everything mention in your guidelines but nothing works.
After that i tried in your sample code and there is no call in background in your sample code too.
You have to look into this seriously.
I know that it's reported many times but i am reporting it for iOS 15.2 and Xcode 13.2. Also for your information i am using this from version 3.x.
@bobiechen-twilio kindly look into the lib and test your sample code for the scenario and then came back to me.
Thanks.


Problem generating mds plot after repOverlap

🐛 Bug

Hi There, I am running into an error while trying to graph with vis after repOverlap, exactly as detailed in the manual
To Reproduce
Steps to reproduce the behavior:
imm_ov1 = repOverlap(immdata.opc$data, .method = "public", .verbose = T)
vis(repOverlapAnalysis(imm_ov1, "mds+kmeans"))

it gives an error
.by="cluster", .meta=immdata.opc$meta
However, when I run
vis(repOverlapAnalysis(imm_ov1, "tsne"))
there is no problem.
This is what the beginning of print(imm_ov1) looks like

I think it might be due to the NAs in the middle. Thoughts?
Thanks in advance for your help

Expected behavior

Additional context

Hi @peter-yufan-zeng
sorry for the late response! Is it possible to replace NAs and try again if it works?
res = repOverlapAnalysis(imm_ov1, "mds+kmeans")
res[is.na(res)] = 0

Ping about the repOverlap MDS issues @peter-yufan-zeng

Replacing the NA's still throws error
'Error in MASS::isoMDS(.data, k = 2, trace = F) :
zero or negative distance between objects 1 and 2'

Thank you, will look into this as well. Thank you a lot for coming back and helping us improve the package, @peter-yufan-zeng !

@vadimnazarov  no problem. I am a huge fan of the package and appreciate your help 👍

Having the same issue, thanks!

Same for me.

Same for me.

Hi,
I am having the same issue. Is there any solution for this error?
Thanks!
Parvathi.

Hi, I tried to reproduce it on the dev version of immunarch and test data data(immdata), but I don't have any problems. Could you share the overlap matrix here? What version of the package are you using?

@vadimnazarov Hi I guess it is caused by the zero value in the overlap matrix, you can try overlap matrix with multiple 0 to reproduce the error


Add ability to set hostname via DHCP during install

Problem:
We need to provide the option of allowing system hostnames to be set via DHCP.
Solution:
When installing Harvester, if the management interface is configured via DHCP, we will default the system hostname to the client hostname specified by the DHCP server.
For interactive installation, the order of the "Configure hostname" and "Configure network" pages is swapped, so that "Configure network" comes first.  This allows us to set DHCLIENT_SET_HOSTNAME="yes" when applying the management NIC config.  If DHCP is used, and the DHCP server is configured to specify a hostname, the live environment will then automatically have its hostname set to whatever it got from the DHCP server.  Then, on the hostname page, we can default the hostname to that value, which the user is still free to change should they wish.
For automated/PXE installation, it works like this:

If os.hostname is specified in the harvester config, we use that.
If os.hostname is not specified, we use the DHCP provided hostname if set, otherwise we fall back to a randomly generated hostname.

(This is an alternative to https://github.com/harvester/harvester-installer/pull/717 - the advantage with this approach is that we set the hostname once, during installation, whereas #717 potentially allows the DHCP server to change the hostname of an installed system later, which would likely cause all sorts of trouble)
Related Issue:
https://github.com/harvester/harvester/issues/1444
Test plan:

Configure a DHCP server, but don't have it set to provide client hostnames (i.e. don't use option host-name XXX or use-host-decl-names on in dhcpd.conf)
Boot the installer ISO and walk through the installation process, selecting DHCP for the management interface.
Verify that the network page now comes before the hostname page, but that the hostname by default remains blank.
Verify that installation completes successfully
Do a PXE install, set to use DHCP, and verify that it completes as usual.
Reconfigure the DHCP server to provide client hostnames.
Boot the installer ISO and walk through the installation process, selecting DHCP for the management interface.
Verify that when you get to the hostname page, the hostname defaults to whatever was specified in the DHCP server config for this host, but that you can also still change the hostname if you want.
Verify that the install completes successfully
Do a PXE install, but make sure os.hostname is not set in the harvester config yaml.
Verify that the installation completes successfully, and that the system picks up the DHCP-provided hostname.

Example dhcpd.conf snippet which should set the hostname to "harv-0-dhcp" for the MAC address 02:00:00:0D:62:E2:
host harvest_node_0 {
    hardware ethernet 02:00:00:0D:62:E2;
    fixed-address <IP_ADDRESS>;
    option host-name "harv-0-dhcp";
}

re: those last two commits, printing the yip output means we get some sort of error message if we can't apply the network config, e.g.:

...and re-ordering the config merge means we can correctly handle remote config that specifies MAC addresses but not interface names.


Replaced borderSide attribute with border attribute in BarChartRodData

Resolve issue #1194
Replaced borderSide attribute with border attribute in BarChartRodData
This enables specifying each border side separately.

I think you need to write some unit-tests

It seems there are some conflicts, please rebase your branch again

I'm sorry for not getting back to you sooner, but I was away for the past two weeks.
I resolved the conflicts. please, try again now.

Hi, sorry for my late response. There are new conflicts.
It would be great if you fix them again.
Thanks!

I resolved recent conflicts. Please provide me with some insights on why these conflicts happen so I can avoid that in the future.
Thanks in advance. @imaNNeo

There is an issue with BarSample 5
Please check it out, It will not happen in the master branch.
But it happens in your branch.
(There might be an issue with the canvas in your changes)
https://github.com/imaNNeo/fl_chart/assets/7009300/154cc07c-40e5-42c2-8851-dc9e3690a282

I resolved recent conflicts. Please provide me with some insights on why these conflicts happen so I can avoid that in the future. Thanks in advance. @imaNNeo

Because we are constantly update the master branch (by pushing new commits). So you need to update your branch with the master.
The best way to do that is rebasing. Please read our contributing guidelines


chore: Update cargo-dist to version 0.17.0

Changes the version of cargo-dist used by Hipcheck to version 0.17.0. Makes changes in release.yml to make it compatible with that version.

@alilleybrinker Take a look at the changes to release.yml and let me know where we might be throwing the baby out with the bathwater.


Deprecate COBOT_PREFIX

Deprecate the use of the environment variable COBOT_PREFIX and
replace it with BOT_PREFIX, issue a warning if COBOT_PREFIX is
still being used.
Closes https://github.com/coala/corobo/issues/370
Reviewers Checklist

[ ] Appropriate logging is done.
[ ] Appropriate error responses.
[ ] Handle every possible exception.
[ ] Make sure there is a docstring in the command functions. Hint: Lookout for
botcmd and re_botcmd decorators.
[ ] See that 100% coverage is there.
[ ] See to it that mocking is not done where it is not necessary.

@yashovardhanagrawal  yeah that sound better. Will do the needful. Any other changes in the code ?

use git grep COBOT_PREFIX and see if its missing to replace it somewhere in the repository

It isn't I checked that

do the needful, I will review it then :smile:

Thanks a lot :+1:

Hey @yashovardhanagrawal  made the changes :+1:

Check line 72 in README.md

@jayvdb  made the changes

ack https://github.com/coala/corobo/pull/505/commits/7a9e09b0f5b7eb66c6f4b2a711d2d23069f7a338

ack https://github.com/coala/corobo/pull/505/commits/7a9e09b0f5b7eb66c6f4b2a711d2d23069f7a338

@gitmate-bot ff


Dark mode problem

Dark mode is not working and button for change is not exists

Switch to dark  has been fixed by @aliskhanoff


Fail to set source in iOS 18

Checklist

[X] I read the troubleshooting guide before raising this issue
[X] I made sure that the issue I am raising doesn't already exist

Current bug behaviour
I try to set the external source as I was working and this error appears in simulator with iOS 18. Then I opened again simulator with 17.5 version and works perfectly, so there is some issue with the new iOS version.
`
import 'dart:async';
import 'package:audioplayers/audioplayers.dart' as ap;
import 'package:dependencies/dependencies.dart';
class AudioPlayerWidget extends StatefulWidget {
/// Path from where to play recorded audio
final String source;
/// Callback when audio file should be removed
/// Setting this to null hides the delete button
final VoidCallback onDelete;
final bool showDeleteIcon;
final bool getFromUrl;
const AudioPlayerWidget({
super.key,
required this.source,
required this.onDelete,
required this.showDeleteIcon,
required this.getFromUrl,
});
@override
AudioPlayerState createState() => AudioPlayerState();
}
class AudioPlayerState extends State {
static const double _controlSize = 56;
static const double _deleteBtnSize = 24;
final _audioPlayer = ap.AudioPlayer()..setReleaseMode(ReleaseMode.stop);
late StreamSubscription _playerStateChangedSubscription;
late StreamSubscription<Duration?> _durationChangedSubscription;
late StreamSubscription _positionChangedSubscription;
Duration? _position;
Duration? _duration;
@override
void initState() {
_playerStateChangedSubscription =
_audioPlayer.onPlayerComplete.listen((state) async {
await stop();
});
_positionChangedSubscription = _audioPlayer.onPositionChanged.listen(
(position) => setState(() {
_position = position;
}),
);
_durationChangedSubscription = _audioPlayer.onDurationChanged.listen(
(duration) => setState(() {
_duration = duration;
}),
);
_audioPlayer.setSource(_source);

super.initState();

}
@override
void dispose() {
_playerStateChangedSubscription.cancel();
_positionChangedSubscription.cancel();
_durationChangedSubscription.cancel();
_audioPlayer.dispose();
super.dispose();
}
@override
Widget build(BuildContext context) {
return LayoutBuilder(
builder: (context, constraints) {
return Column(
mainAxisSize: MainAxisSize.min,
children: [
Row(
mainAxisSize: MainAxisSize.max,
mainAxisAlignment: MainAxisAlignment.spaceBetween,
children: [
_buildControl(),
_buildSlider(constraints.maxWidth),
if (widget.showDeleteIcon) ...[
IconButton(
icon: const Icon(Icons.delete,
color: ThemeColors.indicatorRed, size: _deleteBtnSize),
onPressed: () {
if (_audioPlayer.state == ap.PlayerState.playing) {
stop().then((value) => widget.onDelete());
} else {
widget.onDelete();
}
},
),
],
],
),
if (_duration != null) ...[
LabelWidget(
alignment: Alignment.center,
title: '${prettyDuration(_duration!)}'),
],
],
);
},
);
}
String prettyDuration(Duration duration) {
var seconds = (duration.inMilliseconds % (60 * 1000)) / 1000;
if (duration.inMinutes != 0) {
return '${duration.inMinutes}m ${seconds.truncate()}s';
} else {
return '${seconds.truncate()}s';
}
}
Widget _buildControl() {
Icon icon;
Color color;
if (_audioPlayer.state == ap.PlayerState.playing) {
  icon = const Icon(Icons.pause, color: ThemeColors.myDarkColor, size: 30);
  color = ThemeColors.myDarkColorTerciary;
} else {
  final theme = Theme.of(context);
  icon = const Icon(Icons.play_arrow,
      color: ThemeColors.myDarkColor, size: 30);
  color = ThemeColors.myDarkColorTerciary;
}

return ClipOval(
  child: Material(
    color: color,
    child: InkWell(
      child:
          SizedBox(width: _controlSize, height: _controlSize, child: icon),
      onTap: () {
        if (_audioPlayer.state == ap.PlayerState.playing) {
          pause();
        } else {
          play();
        }
      },
    ),
  ),
);

}
Widget _buildSlider(double widgetWidth) {
bool canSetValue = false;
final duration = _duration;
final position = _position;
if (duration != null && position != null) {
  canSetValue = position.inMilliseconds > 0;
  canSetValue &= position.inMilliseconds < duration.inMilliseconds;
}

double width = widgetWidth - _controlSize - _deleteBtnSize;
if (widget.showDeleteIcon) {
  width -= _deleteBtnSize;
} else {
  width += _deleteBtnSize;
}

return SizedBox(
  width: width,
  child: Slider(
    activeColor: ThemeColors.myDarkColorTerciary,
    inactiveColor: widget.showDeleteIcon
        ? ThemeColors.myDarkColorSecondary
        : ThemeColors.myDarkColor,
    onChanged: (v) {
      if (duration != null) {
        final position = v * duration.inMilliseconds;
        _audioPlayer.seek(Duration(milliseconds: position.round()));
      }
    },
    value: canSetValue && duration != null && position != null
        ? position.inMilliseconds / duration.inMilliseconds
        : 0.0,
  ),
);

}
Future play() => _audioPlayer.play(_source);
Future pause() async {
await _audioPlayer.pause();
setState(() {});
}
Future stop() async {
await _audioPlayer.stop();
setState(() {});
}
Source get _source => widget.getFromUrl
? ap.UrlSource(widget.source)
: ap.DeviceFileSource(widget.source);
}
`
PlatformException(DarwinAudioError, Failed to set source. For troubleshooting, see https://github.com/bluefireteam/audioplayers/blob/main/troubleshooting.md, AVPlayerItem.Status.failed on setSourceUrl: Unknown error, null)<…>
3
[ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(DarwinAudioError, Failed to set source. For troubleshooting, see https://github.com/bluefireteam/audioplayers/blob/main/troubleshooting.md, AVPlayerItem.Status.failed on setSourceUrl: Unknown error, null) import 'dart:async';
Expected behaviour
It should load the external source.
Steps to reproduce

Execute flutter run on the code sample
...
...

Code sample

Code sample
void main() {
}

Affected platforms
iOS
Platform details
No response
AudioPlayers Version
6.1.0
Build mode
No response
Audio Files/URLs/Sources
No response
Screenshots
No response
Logs

my relevant logs

Full Logs

my full logs or a link to a gist

Flutter doctor:
Output of: flutter doctor -v

Related issues / more information
No response
Working on PR
no way

Up, i have the same issue

@rxlabz is there any fix schedule of this?

Can someone add this f15b414135039b72760a908b95c01aa8c9da6422 so we don't get 'Unknown error'

Experiencing the same, the first times it works, but after setting the URL source on consecutive sounds, like 6 times usually it fails.


n Queen in C++

Programming language

[ ] C
[x] C++
[ ] Java
[ ] Python
I want to work on this issue.

It is a Backtracking Algorithm(Hard).

@Ayush7614 please assign this issue to me


QUESTION: NO CHANGE in Philippine Education due to SMUGGLER/CARNAPPER group?

QUESTION: NO CHANGE in Philippine Education due to SMUGGLER/CARNAPPER group?
observed: SMUGGLER/CARNAPPER group has been misusing wireless broadcasting technology for over 18 years;
--> expects: difficulty for group to CHANGE techniques and existing contacts
notes: however, if SMUGGLER/CARNAPPER group cannot CHANGE,
CHANGE in Philippines CAN and does occur;
Example: History of Education in the Philippines
where: after the Second World War, the Philippines
--> had previously been classified to be @TOP in ASIA;
CAUSE: of CHANGE for the worse?
EXCESS MONEY, without know-how,
--> i.e. FAKE/FALSE knowledge from LIARS;
Example: How to build vehicles?
...? (SMUGGLER/CARNAPPER group: "SMUGGLE!")
output: use same ENGINE of small jeep for more carry load;
Reference: JAPAN case, where: in POVERTY, albeit with know-how;
where: FACTORIES and people with know-how to build jet fighters (vehicles) NOT all destroyed;
Example: How to build vehicles?
reuse the FACTORIES and people with know-how;
output: build ENGINE applicable for need (carry load)
ACTION ITEMS

DESTROY SMUGGLER/CARNAPPER group

SHARE KNOW-HOW


Add semantic metric type to metrics

Examples:
Rolling Sum
Rolling Percentile
Cumulative Sum
Rolling Max
(Will fill this in with more once I'm confident I've covered all cases)
This should allow for metrics publishers to be written more generically.  Rather than encode knowledge into each specific example that rollingCountSuccess is a rolling sum, encode it directly in hystrix-core, and let the metrics publisher map that to the proper method of serializing the data

See #854 for more context


Update check-jsonschema usage

Update to v0.19.2 (latest)
Add check-dependabot to validate dependabot config

👋 Hi there, check-jsonschema maintainer here!
I received a PR in reference to #178 on this repo, and the relevant fix has already been included and released in the latest check-jsonschema version (v0.19.2). I noticed that this repo is on a somewhat older version of the hook, so it seemed like a nice/friendly thing to drop a PR with the relevant update.
I recommend using pre-commit autoupdate regularly, or using pre-commit.ci for its autoupdate functionality, as it will help you stay up-to-date.

@sirosen thank you very much, you made my day!

you made my day!

And now you've made my day!
Always a pleasure to be able to help out a stranger. Please drop by and let me know if you ever run into other issues with check-jsonschema!