High CPU usage when sitting idle

When I use the firefly-cli to create a new stack, as soon as the stack comes up, before the user interacts with the system at all, the CPU usage of both firefly-tokens-erc1155 and firefly-ethconnect is very high. This causes my machine to run hot, loud, and slow. If I run the same stack without a token connector ethconnect idles at < 1% CPU usage, so I suspect it's some interaction caused by the token connector that is causing the spike.
CONTAINER ID   NAME                    CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O         PIDS
961488d1f524   test_tokens_0_1         86.86%    52.25MiB / 15.54GiB   0.33%     216MB / 229MB     0B / 4.1kB        22
e9dba6fdd5ba   test_tokens_1_1         87.37%    53.53MiB / 15.54GiB   0.34%     215MB / 228MB     643kB / 4.1kB     22
2a9f77c7c4ee   test_ethconnect_0_1     78.51%    16.38MiB / 15.54GiB   0.10%     229MB / 216MB     0B / 12.3kB       15
678af5c8b919   test_ethconnect_1_1     77.72%    13.96MiB / 15.54GiB   0.09%     228MB / 215MB     0B / 12.3kB       14

Seems to be caused by #14


K9 Mail error: Failed to send some messages - Negative SMTP reply: 553

I recently started getting the following error when trying to send an email on my android device via K9 mail:
Failed to send some messages - Negative SMTP reply: 553
The issue consistently happens on my zoho mail account, and only occasionally on my Gmail account as well.
Expected behavior
when I send and email, it should be successfully sent and should show up in the sent folder
Actual behavior
message does not appear in either the "sent" folder or the "outbox" folder
message shows up in the "drafts" folder and then disappears upon refresh of the "drafts" folder
Steps to reproduce

compose an email in K9 mail app using Zoho mail account
click "Send"

Environment
K-9 Mail version: 5.207
Android version: 6.0
Account type (IMAP, POP3, WebDAV/Exchange): IMAP

I found a workaround by deleting the zoho mail account from K9 app and setting it up over again with exact same settings - issue goes away


issues

there isn't ++ for adding + 1 to the variable
REPL just quit if there is an error, just implement a built in error system
can't fix code by moving cursor

can't edit objects (obj.val = "test"; throws an error)

doesn't exit after execution
enters REPL mode on error

backslash is not implemented

there is no return to functions

there is no arrays

remove GeoIP, replace all currencies with $

exec stdout not working
add stdin

add a null value

add docs

object.set() (if that's a thing) just freezes execution on REPL

types not working

types not working

types work in bsx but bussin isn't made as a typescript clone 💀💀💀

types not working

types work in bsx but bussin isn't made as a typescript clone 💀💀💀

so why there is object and string and etc...?

also I recommend you to try my fork, it's better

also I recommend you to try my fork, it's better

All you've done is make tiny changes that arguably make stuff worse

also I recommend you to try my fork, it's better

All you've done is make tiny changes that arguably make stuff worse

I am making stdin 😎 and removed the object from the REPL output 😎

Also I am adding more data types

You're taking this lang too seriously its made as a meme

you don't want to rizz up with errors?

btw, I want to run this 24/7 on a server

You're taking this lang too seriously its made as a meme

I want to make Bussin in Bussin so it's gonna be meme squared


Pub Sub Polling interval for Kafka

The subscriber polls to get the messages from subscription topic in kafka. The interval at which request is made to get the new message is not configurable as of now. Making this parameter configurable will help in tuning the  application for performance needs.
@pravinpushkar

Moving to correct repo.

There's a huge number of configuration settings for kafka consumers and producers. E.g. a couple more that we use are partition.assignment.strategy and client.dns.lookup.
Ideally all these settings should be configurable, instead of just a very limited fixed subset of them.

/keep-alive

Currently Pub-sub configurations are handled by sarama library[https://pkg.go.dev/github.com/shopify/sarama] and there is not much scope for the App developers to edit the configurations. I think extending the implementation to allow configuration edits may help.

It'd be REALLY hard to decide which subset of config. settings should be supported and which should not, as they all are important in different situations. If the goal is to support developers moving from kafka-specific clients that support all config. settings (which it should be), I'd suggest allowing and passing through all config. settings. Suggest having them in a separate section in the config. file vs. having them mixed in with the dapr config. settings.

If there is a way to pass through the settings the best we can probably do is providing a single metadata field which would be a comma separated list of key value pairs. And you'd have to refer to the Sarama library to get a list of supported settings. We will not separately document the supported options, but we can probably link to it from the docs.
Still I do not know whether this is possible.
By the way, as a maintainer of the components I do not think that the goal is for those to replace the use of native clients for advanced use cases / configuration scenario. Dapr is an abstraction and you always lose some ability to customize things with each abstraction.

/ keep-alive

/ keep-alive

We need to have provision to tune the kafka parameters. There are many configurable options provided by sarama library, but only few options are available via dapr. Kafka is critical piece in our architecture and based on the load, message size we will need to tune these params as per needs. Hence requesting to provide configurable parameters to subscribers/publishers. While the dapr can continue to keep the defaults as it is today, however consumers should be able to override them.
Below is params from library which can be made configurable
c.Producer.MaxMessageBytes = 1000000
c.Producer.Timeout = 10 * time.Second
c.Producer.Partitioner = NewHashPartitioner
c.Producer.Retry.Max = 3
c.Producer.Retry.Backoff = 100 * time.Millisecond
c.Producer.Return.Errors = true
c.Producer.CompressionLevel = CompressionLevelDefault
c.Producer.Transaction.Timeout = 1 * time.Minute
c.Producer.Transaction.Retry.Max = 50
c.Producer.Transaction.Retry.Backoff = 100 * time.Millisecond
c.Consumer.Fetch.Min = 1
c.Consumer.Fetch.Default = 1024 * 1024
c.Consumer.Retry.Backoff = 2 * time.Second
c.Consumer.MaxWaitTime = 500 * time.Millisecond
c.Consumer.MaxProcessingTime = 100 * time.Millisecond
c.Consumer.Return.Errors = false
c.Consumer.Offsets.AutoCommit.Enable = true
c.Consumer.Offsets.AutoCommit.Interval = 1 * time.Second
c.Consumer.Offsets.Initial = OffsetNewest
c.Consumer.Offsets.Retry.Max = 3
c.Consumer.Group.Session.Timeout = 10 * time.Second
c.Consumer.Group.Heartbeat.Interval = 3 * time.Second
c.Consumer.Group.Rebalance.GroupStrategies = []BalanceStrategy{BalanceStrategyRange}
c.Consumer.Group.Rebalance.Timeout = 60 * time.Second
c.Consumer.Group.Rebalance.Retry.Max = 4
c.Consumer.Group.Rebalance.Retry.Backoff = 2 * time.Second
c.Consumer.Group.ResetInvalidOffsets = true
c.ChannelBufferSize = 256

We need to have provision to tune the kafka parameters. There are many configurable options provided by sarama library, but only few options are available via dapr. Kafka is critical piece in our architecture and based on the load, message size we will need to tune these params as per needs. Hence requesting to provide configurable parameters to subscribers/publishers. While the dapr can continue to keep the defaults as it is today, however consumers should be able to override them.
Below is params from library which can be made configurable
c.Producer.MaxMessageBytes = 1000000
c.Producer.Timeout = 10 * time.Second
c.Producer.Partitioner = NewHashPartitioner
c.Producer.Retry.Max = 3
c.Producer.Retry.Backoff = 100 * time.Millisecond
c.Producer.Return.Errors = true
c.Producer.CompressionLevel = CompressionLevelDefault
c.Producer.Transaction.Timeout = 1 * time.Minute
c.Producer.Transaction.Retry.Max = 50
c.Producer.Transaction.Retry.Backoff = 100 * time.Millisecond
c.Consumer.Fetch.Min = 1
c.Consumer.Fetch.Default = 1024 * 1024
c.Consumer.Retry.Backoff = 2 * time.Second
c.Consumer.MaxWaitTime = 500 * time.Millisecond
c.Consumer.MaxProcessingTime = 100 * time.Millisecond
c.Consumer.Return.Errors = false
c.Consumer.Offsets.AutoCommit.Enable = true
c.Consumer.Offsets.AutoCommit.Interval = 1 * time.Second
c.Consumer.Offsets.Initial = OffsetNewest
c.Consumer.Offsets.Retry.Max = 3
c.Consumer.Group.Session.Timeout = 10 * time.Second
c.Consumer.Group.Heartbeat.Interval = 3 * time.Second
c.Consumer.Group.Rebalance.GroupStrategies = []BalanceStrategy{BalanceStrategyRange}
c.Consumer.Group.Rebalance.Timeout = 60 * time.Second
c.Consumer.Group.Rebalance.Retry.Max = 4
c.Consumer.Group.Rebalance.Retry.Backoff = 2 * time.Second
c.Consumer.Group.ResetInvalidOffsets = true
c.ChannelBufferSize = 256


Question: should I merge my examples in?

I have some example code that I'd love to donate to the project to help folks learn react-server, but I'm not sure if it's something that y'all want and/or where it would go.
@gigabo: in this comment, you said that "[t]hose are definitely better showcase examples than what I'm imagining here. I think they make sense to maintain as separate packages". I wasn't sure if that means you want me to keep them as separate packages in a separate repo, or that you mean they'd be welcome as sub-packages of the react-server monorepo. (I'm fine either way.)
So I'm looking for thoughts on if these would be a good addition to the monorepo, and if so, where they should go in the folder structure. Thanks!

Oh, I meant a separate package in the monorepo!  packages/react-server-examples.  Bring it!

Closing, I got my answer and filed a PR. Thanks!


WIP: Windows on Arm support

PR Details
This is a highly experimental WIP draft of getting the entire Stride suite to compile and run on Windows on Arm (WoA).
There has been a lot of broad stroke changes that change projects from win-x64 to win-arm64, just to test this proof of concept.
I have gone through every mention of x86/x64 through the repo to add in arm64 where appropriate, but there are some issues surrounding the native libraries. I feel that there's perhaps documentation somewhere on how to build these binaries that exist in the repo, maybe?
I am seeking any and all feedback and assistance with this PR where possible. I am on Discord in the Stride server as pierce @archanox
Milestone:

Getting the solution to build for arm64 with everything working (tests, packaged, installer built, etc)
Configure multiple targets for a win-x64 build and win-arm64

Current Issues checked in:

ClearScript was only bundled with x86/x64 binaries. Dependencies for the Metrics were erroneously upgraded for the entire project. Untested.
There are missing binaries for arm64 that do not exist in the repo
32bit/64bit checks assume x86 platform
Native library build scripts appear to be pointing at an old version of visual studio
MSBuild paths for x86/x64 have been temporarily removed until successful build & test.
Preliminary ARM64 UWP changes were checked in, but will likely be reverted.

Goal of this PR:

To develop on a WoA device, with Visual Studio integration and to build for Windows and Linux platforms.

Outside the scope of this PR:

CI for automated builds of the arm64 version
NuGet publishing

Related Issue
#2018
Types of changes

[x] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] My change requires a change to the documentation.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.
[ ] I have built and run the editor to try this change out.

@dotnet-policy-service agree

Was there any more movement on this (or discussion on other platforms)? Was there anything preventing review?

Was there any more movement on this (or discussion on other platforms)? Was there anything preventing review?

Hi Anthony, no movement I'm afraid. I've been pretty intimidated by Stride's bespoke build system.
At the time of me authoring this PR, there was still a lot of native dependencies that are precompiled and checked into the repo.
The last thing I believe I got stuck on was actually building the dependencies and locating where they need to live. MSBuild wasn't very transparent to what the cause was, merely just reporting what broke because of it.
As for review, if anyone else is interested in this PR, I'd advise just spinning up a new one without the mess I made when making the entire solution build and run just for aarch64 rather than x64.
If you are interested in picking this up and wish to avoid reading the mess in the singular commit, I think the main take aways are that

the MSBuild location in the registry is in another location.
The version of MSBuild that Stride targeted when I looked last was a version prior to a native WoA version of it.
And the Url referenced to download MSBuild for arm64 is actually a fat binary for both x64 and arm64.


Update 10-kubernetes.md

one more spelling fix.
great write-up by the way. 👍

@ajschmidt8 awesome, thank you!


ISSUE-153 Schema compatibility validation

Closes #153

+1 LGTM

+1

Thanks @satishwaghela , amended commit message and merged it.


Replace tm_tensor with torch-mlir proper

Per discussion on iree-discuss, this patch removes the original tm_tensor source snapshots and special casing from the codebase, instead pulling in the parts of torch-mlir that provide a full solution, starting from the torch dialect down.
This reuses the existing IREE_INPUT_TORCH CMake define but routes it to a compiler plugin at compiler/plugins/input/Torch where all build plumbing to integrate and local pipelines exist. I have kept the original tm_tensor input type for the moment since downstreams are using that, but we will want to eventually replace that with a dedicated torch input type which works directly from the torch dialect.
This patch should be NFC to all current users, however it allows programmatic and API access to the full Torch dialects and conversion pipelines, which is essential for broader scale integration.
We add third_party/torch-mlir as a submodule at torch-mlir's current HEAD. During LLVM integrates, if there are ever API breaks that affect this, we may need to push patches to torch-mlir to address. This can be done on a torch-mlir branch as needed (or committed at head). I have been tracking changes for a couple of months and have not had a need to do this yet, which indicates that this is relatively stable.
Committing with an XFAIL'd lit test, which for some reason was not running previously and now is: https://github.com/openxla/iree/issues/14916

Yes, it is entirely self contained as compiler/plugins/input/Torch
This should be an improvement for such cases over the previous state where it was optional but still threaded through the main codebase.

Ah... I just noticed that if someone runs git submodule update --init --recursive, that will now also clone torch-mlir's copy of llvm-project. Recursive updates are less common when working directly in IREE, but downstream projects may use it for convenience vs explicitly choosing which subfolders and submodules to use.
Not sure how we could improve the situation there... maybe write down the git submodule workflows we suggest and keep an eye on downstream projets (e.g. https://github.com/RechieKho/IREE.gd#build-from-source)?

Ah... I just noticed that if someone runs git submodule update --init --recursive, that will now also clone torch-mlir's copy of llvm-project. Recursive updates are less common when working directly in IREE, but downstream projects may use it for convenience vs explicitly choosing which subfolders and submodules to use.
Not sure how we could improve the situation there... maybe write down the git submodule workflows we suggest and keep an eye on downstream projets (e.g. https://github.com/RechieKho/IREE.gd#build-from-source)?

The instructions at https://github.com/iree-org/iree-template-runtime-cmake/ should also be updated to account for that too:

For a faster checkout the LLVM dependency can be dropped as this template is only compiling the runtime (only bother if optimizing build bots):
$ git \
    -c submodule."third_party/llvm-project".update=none \
    submodule update --init --recursive

Output even with that (https://github.com/iree-org/iree-template-runtime-cmake/actions/runs/6656984452/job/18090744983):
Submodule path 'third_party/iree/third_party/torch-mlir': checked out '6f81ad72938deb56c6d43bbc01388c1f8f1253c1'
Submodule 'externals/llvm-project' (https://github.com/llvm/llvm-project.git) registered for path 'third_party/iree/third_party/torch-mlir/externals/llvm-project'
Submodule 'externals/stablehlo' (https://github.com/openxla/stablehlo.git) registered for path 'third_party/iree/third_party/torch-mlir/externals/stablehlo'
Cloning into '/home/runner/work/iree-template-runtime-cmake/iree-template-runtime-cmake/third_party/iree/third_party/torch-mlir/externals/llvm-project'...


Design/control utils/responsibility shift

The last PR so far related to the ControlUtils class. All responsibilities that should be handled by the Roboteam_utils repository are removed and placed in Roboteam_utils instead. Note that you should only merge this PR, once Design/control utils shift #67 on the Roboteam_utils branch is merged. Moreover Design/control utils/codeclone removal #1060 should also be merged first. And this PR should be simultaneously merged with Fix bug in ShortestAngleDiff #68 on the Roboteam_utils repository.

It looks like there is a lot of functionality that is being removed from controlutils but not put back in roboteam_utils. I would like to hear what @jordiluong and @rolfvdhulst think about this change

It looks like there is a lot of functionality that is being removed from controlutils but not put back in roboteam_utils. I would like to hear what @jordiluong and @rolfvdhulst think about this change

The functionality is not removed. All dead functions are placed at the bottom of the file. Though the cloned functions are removed.


[REQUEST] - Question about private collection

Reference Issues
#177 follow-up
Summary
@taprosoft

You can go to Resources > File Collections > set private: true to restrict access to only owner of that file.

One issue is that only admins has Resources menu, it seems a user can't create private collection for its' own.
When admin creates a private collection, it isn't not linked to a user, so: everyone can see it, and can upload into. I agree that access control still has place - only those who upload into a private should only see it.
IMHO ideally would be

assign private collection to a user
hide it from others
let users (non-admin) create private collections themselves. (Is it blocked by necessity of restating app after creating collection?)

Does it make sense? How hard to reach those aims?
Basic Example
There's a private collection assigned to someone, what others can't see this collection.
Drawbacks
Maybe it's too hard.
Additional information
No response

assign private collection to a user
hide it from others
let users (non-admin) create private collections themselves. (Is it blocked by necessity of restating app after creating collection?)

Thanks for the suggestion. Indeed this use case is valid from our perspective. We are working on a "lite" collection feature called file group, which help users to define their own custom set of files and perform QA on it, or, combine any file group with individual files.
Let me know if it fit your request @mkhludnev.

We are working on a "lite" collection feature called file group

It sounds like a right direction toward. But it's not obvious why to introduce a new entity. I don't know, but maybe it's related to limitations of existing collections - I noticed required reboot message. I already described how it seems to me, but you definitely have a vision and can choose a better approach.
Thanks for the great project!

moving to https://github.com/Cinnamon/kotaemon/discussions/241


When the king is in check it should become red

As an indication to the player that it's in check.

Done.


refactor(kv-redis): use provider-wit-bindgen

Feature or Problem

Groundwork for adapting the provider-wit-bindgen in kvredis provider
Note, that we cannot match the identifier casing to Smithy - Smithy interfaces and structs are using PascalCase, while fields are using camelCase. Ideally, I would expect the bindgen to have a parameter to select the capitalization, but we could just hardcode it for now
Related Issues

#662
#470
Release Information

Consumer Impact

Testing

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin
[ ] x86_64-windows

Tested on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin
[ ] x86_64-windows

Unit Test(s)

Acceptance or Integration

Manual Verification

@rvolosatovs close this PR, the functionality has landed


JAV extension request

Describe your suggested feature
Knowing that there's plenty of hentai anime extension, I wonder if we could have a JAV site extension as well?
Particularly an extension to the missav.com website where full duration JAVs are posted there. Sadly that website didn't have the best control buttons for the video. I think aniyomi could cover up for that.
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with an extension, I should be opening an issue in the extensions repository.
[X] I have updated the app to version <IP_ADDRESS>.
[X] I will fill out all of the requested information in this form.

Please open an issue in the extensions repository instead.


suggested expansion to definition for GO:0030254

type III protein secretion system (GO:0030254 ) --
suggest that the definition be expanded -- what is
there is not incorrect, but more infomration would be
useful.
Suggest expanded definition: "A bacterial secretion
system in which secretion occurs in a continuous
process without the distinct presence of periplasmic
intermediates; does not involve proteolytic processing
of secreted proteins; several components of the system
have homologs involved in flagellar biogenesis; the
system is capable of injecting proteins into eucaryotic
cells."
Reference supporting this definition is PMID =
11018143
Reported by: ccollmer
Original Ticket: "geneontology/ontology-requests/1784":https://sourceforge.net/p/geneontology/ontology-requests/1784

thanks www.movied.org


database

Do we have the database for this dataset?

Our dataset is publicly available at https://huggingface.co/datasets/zhanghanchong/css. You can download the databases.


2022 Day 13 retrospective

We could replace the mask/unmask magic here https://github.com/jp7677/adventofcode/blob/main/2022/src/test/kotlin/Day13.kt#L17 by just walking sequentially trough the string and and pick up either integer or list (by jumping until the closing bracket).

Done in https://github.com/jp7677/adventofcode/commit/aae031a2cb864741f30fdb374e29adbd14834008


Refactor caching layers in dynamo collector

I wanted to better understand what was going on, so I refactored it.
Highlights:

S3 client code moved from dynamo_collector into a new file, and given same interface as MemcacheClient
NewDynamoDBCollector now takes both a memcache client and an s3 client, rather than being responsible for constructing both
FetchReports returns a map of fetched reports, making it easier to write back to the in-process cache

Lowlights:

We write back to the in-process cache after retrieving from the in-process cache
memcache constants now in prog/app.go
despite branch name, doesn't actually instrument

This change is

PTAL.

LGTM!


Scaling up the interceptor

Scaling up the interceptor (along with the app pods) on high traffic events.
Checklist

[x] Commits are signed with Developer Certificate of Origin (DCO)

Fixes #30

@khaosdoctor @zroubalik @tomkerkhove can one or more of you PTAL?

I'll leave it up to the others

@zroubalik @tomkerkhove thanks


⚠ Jenkins Plugins Artifactory has degraded performance

In 58ed6f1, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1246 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 909e7f2 after 49 minutes.


[feature request] Support reading equality delete files

Feature Request / Improvement
Only position delete is supported right now
https://github.com/apache/iceberg-python/blob/e5a58b34dd830c6ffea11649613b693f70f7cbb4/pyiceberg/table/__init__.py#L1418
Let's also add reading equality delete
Position delete PR https://github.com/apache/iceberg/pull/6775

Thanks @kevinjqliu, I can work on this issue

This will be a fantastic addition to PyIceberg! Thank you for raising this issue @kevinjqliu and @Zyiqin-Miranda 🎉

Thanks @kevinjqliu and @sungwy. Starting to add support to current plan_files function for equality deletes, not sure if the current _InclusiveMetricsEvaluator can be directly used to determine whether the equality delete files is relevant to the data files?
Seems like Iceberg Java uses canContainEqDeletesForFile instead.
My understanding is that position deletes can use lower_bound == upper_bound of file_path column to filter out irrelevant files quickly but equality deletes don't have this advantage, so basically equality deletes can be relevant to any data files within same partition. Thanks for any insights here in advance!

Equality Delete Files and Scan Planning are good docs for this.
My general understanding is that equality deletes are applied to all data files (across all partitions, if partitioned).

Position delete files must be applied to data files from the same commit, when the data and delete file data sequence numbers are equal. This allows deleting rows that were added in the same commit.


Cell temperature equals ambient temperature when using weather file POA

Describe the bug
I have a custom weather file with measured POA data. If I use the heat transfer method (CEC module model) for temperature, cell temperature is being calculated as equal to ambient temperature.
Expected behavior
I would expect cell temperature to be different from ambient temperature in most daytime hours.
Screenshots

Operating System and Version

OS: Windows
Version: SAM 2022.11.21 r3

Additional context
Temperature seems ok when using the NOCT method.

@williamhobbs Could you share an example of a weather file that causes this results? I was not able to replicate this using our test POA weather file.

Here you go. It's reformatted from "Phase1_meteo_hourly.xlsx" here https://pvpmc.sandia.gov/resources-and-events/2023-blind-modeling-comparisons/. It seems to work ok otherwise, so I think I got the formatting correct.
Phase1_meteo_hourly_sam.csv

Thanks for sharing the file. The Heat Transfer method requires atmospheric pressure and wet-bulb temperature data (see the note in the "Heat transfer method" section of the Module Help topic).
To fix this issue, we will:

Add a simulation error when the heat transfer method is selected and those weather data elements are missing from the file.
Revise the "Weather Data Elements" topic to mention this requirement.


Multiple leg CO2 estimation from single HTTP request

Closes #15
Changes

process multiple CO2 annotations with a single request

@konker here is the WIP pull request for multi-itinerary support.

@konker this is ready for review. I added an example_request.json and instructions for using curl in the README. The same request can be made from the documentation UI.

@konker I made the requested change and fixed the issue causing the 500 error. Please review again at your convenience.

Additional fields on the input leg objects are still not copied through – could you add the **kwargs where I mentioned it above?

Additional fields on the input leg objects are still not copied through – could you add the **kwargs where I mentioned it above?

"
@MarkusShepherd I am not sure that the **kwargs need to be carried through the estimator function. The original leg is preserved in the following manner:
{**get_co2_estimate(**leg), "request_object": leg}

We only pass the leg into the estimate_co2 function in order to produce the co2_estimate object.

I think this results in a bit of a mishmash: we have the leg with estimates but without all the fields, and inside the original leg with the additional fields, but without estimates. As a caller, if what I want is the complete leg with estimates annotated, I need to assemble that information myself.
If this is what we decide it should return, we should remove *args and **kwargs from the get_co2_estimate() signature as they are not used.
There's a couple of other smaller issues you see when running pylint:
api.py:1:0: C0114: Missing module docstring (missing-module-docstring)
api.py:20:0: W1113: Keyword argument before variable positional arguments list in the definition of get_co2_estimate function (keyword-arg-before-vararg)
api.py:21:0: W0613: Unused argument 'args' (unused-argument)
api.py:21:0: W0613: Unused argument 'kwargs' (unused-argument)
api.py:64:0: E0102: function already defined line 5 (function-redefined)
api.py:64:0: C0116: Missing function or method docstring (missing-function-docstring)
api.py:5:0: W0611: Unused estimate_co2 imported from transport_co2 (unused-import)

If this is what we decide it should return, we should remove *args and **kwargs from the get_co2_estimate() signature as they are not used.

As mentioned previously, when I remove the **kwargs from the function signature, I get an error:
TypeError: get_co2_estimate() got an unexpected keyword argument 'other_property'

we have the leg with estimates but without all the fields, and inside the original leg with the additional fields, but without estimates. As a caller, if what I want is the complete leg with estimates annotated, I need to assemble that information myself.

Yeah, that is one downside to the design. I am trying to avoid potential property collisions by returning a dictionary in the following form:

response

co2_estimate: the estimate and related values used to derive the estimate
request_object: the full request object, preserving all of its properties

If this is what we decide it should return, we should remove *args and **kwargs from the get_co2_estimate() signature as they are not used.

As mentioned previously, when I remove the **kwargs from the function signature, I get an error:
TypeError: get_co2_estimate() got an unexpected keyword argument 'other_property'

If we don't want to consume the arguments, we shouldn't pass them to the function, i.e., we should only pass the arguments we really need. This makes things more verbose, of course, but silently catching wrong parameters is worse IMHO. As the zen goes: explicit is better than implicit.

There's a couple of other smaller issues you see when running pylint

OK, the pylint issues are resolved.

we have the leg with estimates but without all the fields, and inside the original leg with the additional fields, but without estimates. As a caller, if what I want is the complete leg with estimates annotated, I need to assemble that information myself.

Yeah, that is one downside to the design. I am trying to avoid potential property collisions by returning a dictionary in the following form:
* response

  * **co2_estimate**: the estimate and related values used to derive the estimate
  * **request_object**: the full request object, preserving all of its properties

I'd lean towards thinking that just returning the original leg object, annotated with co2_estimate (and possibly distance and occupancy as calculated along the way) is the cleaner API, but I can be persuaded either way.

I'd lean towards thinking that just returning the original leg object, annotated with co2_estimate (and possibly distance and occupancy as calculated along the way) is the cleaner API

I agree. I was just trying to play it safe here.
It is also difficult to figure out how to define an object in OpenAPI Specification that can contain

arbitrary fields
oneOf

co2_estimate
error

The co2_ estimate or error property might already exist on the calling object for some reason. I know it is a bit awkward, but am thinking it might be safer to reduce those values on the calling side and handle issues synchronously rather than in the API.

OK, I see. Overall keeping the original leg object unchanged seems the easier and straightforward implementation. In this case all that remains (from the code side) would be to change the call to get_co2_estimate() with explicit parameters.
In the meantime, let me take another look at the documentation to make sure everything is clear.

In this case all that remains (from the code side) would be to change the call to get_co2_estimate() with explicit parameters.

Cool. Would you consider opening a PR or making a code suggested with the relevant changes?

In this case all that remains (from the code side) would be to change the call to get_co2_estimate() with explicit parameters.

Cool. Would you consider opening a PR or making a code suggested with the relevant changes?

Pushed. 😎

@konker can you have a look at this PR? It should enable us to estimate CO2 for multiple legs in one request. Hopefully this will make it possible to use this CO2 estimator in our backend.

Apologies for taking so long with this. I've tried it out locally, and looks good.

Don't worry, it took us a few iterations to get this right. 😎


Feature request: Provide an automatic way to upgrade the format of file

https://github.com/cberner/redb/blob/51ca54cd913a075a3bd0acd8793f198aa6e12c29/src/error.rs#L201
When we encounter this issue in production, it's tricky to walkaround it except deleting the old store entirely. This would cause all existing data lost.

There is example code for upgrading from v1 to v2: https://github.com/cberner/redb/issues/785

Thanks for the link. Is that example part of the redb repo? If redb format gets upgraded again in the future, would that example apply too? I was hoping the redb library can provide an way to upgrade the format automatically. This is especially important for things run in production.

Yes, the approach used in that example should work for future changes, because it reads the data using the public interface of the old version of redb and then writes using the newer redb.
Unfortunately, it was not feasible to provide an automatic upgrade between version 1 and version 2.
redb follows semvar versioning, so you can expect any changes in 2.x to provide automatic upgrades, but changes between 1.x and 2.x to require manual upgrade


perf: cached fs utils

Description
Implements a cached strategy for fs checks when the experimental option server.fs.cachedChecks: true is set.
I tested it with https://github.com/yyx990803/vite-dev-server-perf and resolveId times are cut in half. Also tested with the turbopack triangle benchmark using https://github.com/sapphi-red/performance-compare with the same results on my M1 pro.
An example profile diff for tryResolveRealFileWithExtensions (65ms -> 19ms)
Base: 65ms

Cached: 19ms

Context
We received a CPU profile for server start on a large app that takes a lot of time on Windows old machines, and 40% of the server time was spent resolving ids. In particular in fs.realPathSync.native and tryResolveRealFileWithExtensions.
We heavily optimized resolving during Vite 4.3 (see blog post with explanation). We have tried removing realpath calls at least for some cases before, for example, in the closed https://github.com/vitejs/vite/pull/12818.
We tried to avoid caching fs checks so far, but we changed that for the public dir in:

https://github.com/vitejs/vite/pull/15195

Details
This PR implements a cache for watched files in root (not symlinked). We construct the tree lazily using readdir and withFileTypes: true so for each dirent we know if it is a directory, file, or symlink (could be either). We expand all the dirents in the path on each fs check except for the symlinks. So apart from avoiding fs.existsSync, fs.statSync, etc, we can also avoid calls to realpath for each path we cached in the tree because we know there aren't any symlinks in their path.
The cached checks only run during dev because we need a watcher to add/unlink files and directories. For now, the optimization is disabled if watch is null or if there is a custom watch.ignore. We can later relax this restriction.
We discussed with @sheremet-va, and for Vitest to be able to use this optimization, we need to compose all the cached fs trees into a single one so several servers can share the cache of overlapping roots (important for monorepos). We can work on this in a future PR.

What is the purpose of this pull request?

[ ] Bug fix
[ ] New Feature
[ ] Documentation update
[x] Other

Watching the addition of directories and files is more performant now after https://github.com/vitejs/vite/pull/15279/commits/a343c66979465d8a3e562e869e7cdf9bcc32781a, we don't drop the parent dir dirents and instead add the entry as directory_maybe_symlink or file_maybe_symlink and we resolve it with a lstat lazily if it is ever accessed. We also need this if we want to share the dirents cache for different ResolvedConfigs.

Some details about the updates:

In feat: optimize tryResolveRealFileWithExtensions we avoid many normalizePath and search in the same cached fs tree calls. We could go further and optimize like this the whole tryFsResolve. We can try that in another PR, I think we need better abstractions around handling the DirentCache tree first.
In feat: avoid double normalizePath calls we avoid a few extra normalizePath calls. I saw in profiles that this is starting to take most of the time. If we abstract tryFsResolve we could do a single normalizePath for the whole tree. One issue I see is that I don't know if the fallback fs calls are slower when you pass them normalized paths so we should have both the fs native and the normalized path around? Something else to explore later.
There were fails at one point that I thought were related to Vitest having a custom watch.ignore. I pushed chore: remove watch null and custom ignored guard, and I no longer see these issues. I think we should merge the PR without this guard so more projects can test it out. We still need to review how watch.ignore and watch: null should affect the fs cache
In chore: remove async wip code, I remove some code that was initially intended to let us do async readdir calls. For now, the fs cache is sync and I don't know if we should change that. The readdirSync time is very low compared to everything else. I moved everything to async calls here https://github.com/vitejs/vite/pull/15211 and we didn't see a definite gain either. Maybe a better approach is to keep the tree sync only but auto-expand it in the background doing async readdir calls and assigning the result in sync.

There is also now a version sharing the fs trees, to improve Vitest in monorepo setups. I think we could review and merge that one later:

https://github.com/vitejs/vite/pull/15294

@sapphi-red tested in his Windows machine:

Results with performance-compare on my Windows laptop. It's a 5% improvement
Before: 4202.2ms (babel), 2666.0ms (swc)
After: 4046.8ms (babel), 2491.2ms (swc)
The results with https://github.com/yyx990803/vite-dev-server-perf.
Before
1000 TS modules (50x20) loaded in: 1183ms (runs: [1254,1094,1183,1185,1152])
1000 TS modules (20x50) loaded in: 1054ms (runs: [1126,1075,1052,1035,1054])
2000 TS modules (100x20) loaded in: 2556ms (runs: [2206,2556,2529,2559,2612])
5000 TS modules (250x20) loaded in: 6398ms (runs: [6491,6396,6473,6398,6190])
10000 TS modules (400x25) loaded in: 12524ms (runs: [12954,13004,12437,12437,12524])
After
1000 TS modules (50x20) loaded in: 1067ms (runs: [1130,1020,1012,1076,1067])
1000 TS modules (20x50) loaded in: 1025ms (runs: [1017,1054,1019,1025,1081])
2000 TS modules (100x20) loaded in: 2280ms (runs: [2220,2308,2280,2198,2313])
5000 TS modules (250x20) loaded in: 6253ms (runs: [6282,6253,6023,6131,6267])
10000 TS modules (400x25) loaded in: 12132ms (runs: [12365,12132,12003,11987,12477])

I think my main concern so far is the graph fs eagerness, and that we're still doing a bit of cache-check work for existsSync (and others), that maybe it isn't as bad is we swap it with a isPathWatched check instead?

Could you expand on the isPathWatched idea? About the graph fs eagerness, see https://github.com/vitejs/vite/pull/15279#discussion_r1421431611

About #15294, I'm not really comfortable with doing that and I think we should avoid it 😅 Feels a bit going too far.

I think it will depend on the results that @sheremet-va sees with Vitest. If it speeds up things a lot, I don't think the extra complexity is that bad.

There were fails at one point that I thought were related to Vitest having a custom watch.ignore. I pushed chore: remove watch null and custom ignored guard, and I no longer see these issues. I think we should merge the PR without this guard so more projects can test it out. We still need to review how watch.ignore and watch: null should affect the fs cache

Oh... I accidentally enabled the flag by default in https://github.com/vitejs/vite/pull/15279/commits/a98c9b40ac260478f0b02e81f4ef93e7e10d4d4f#diff-abb3345b6e3b2ec6d297c2ebc54cca85ae4487a31bac3cc9e78457f5114adb26L955, and I see this fail again. The problem is that a file is generated in outDir and I think that isn't being watched. I think we can still release without the guard so we get more feedback.


Error detected in pubspec.yaml when including Flame

Hi @luanpotter,
First of all thanks for making this tool, looks amazing and just what I need.
I'm having an issue when I add Flame as a dependency.
This is how my pubspec.yaml looks like:
dependencies:
  flutter:
    sdk: flutter
  cupertino_icons: ^0.1.0
  cloud_firestore: ^0.0.3
  firebase_auth: ^0.3.1
  flame: ^0.6.2

I run flutter packages get which exits with 0.
Then I run flutter run and I get the output below:
Launching lib/main.dart on iPhone X in debug mode...
Error detected in pubspec.yaml:
#/properties/flutter: type: wanted [object] got null
Running pod install...                                2.8s
CocoaPods' output:
↳
      Preparing

    Analyzing dependencies

    Inspecting targets to integrate
      Using `ARCHS` setting to build architectures of target `Pods-Runner`: (`arm64`)

    Finding Podfile changes
      A audioplayers
      A path_provider
      M cloud_firestore
      - Flutter
      - firebase_auth

    Fetching external sources
    -> Fetching podspec for `Flutter` from `/Users/andreu/flutter/bin/cache/artifacts/engine/ios`
    -> Fetching podspec for `audioplayers` from `/Users/andreu/.pub-cache/hosted/pub.dartlang.org/audioplayers-0.3.0/ios`
    -> Fetching podspec for `cloud_firestore` from `/Users/andreu/.pub-cache/hosted/pub.dartlang.org/cloud_firestore-0.0.4/ios`
    -> Fetching podspec for `firebase_auth` from `/Users/andreu/.pub-cache/hosted/pub.dartlang.org/firebase_auth-0.3.1/ios`
    -> Fetching podspec for `path_provider` from `/Users/andreu/.pub-cache/hosted/pub.dartlang.org/path_provider-0.2.1+1/ios`

    Resolving dependencies of `Podfile`

    Comparing resolved specification to the sandbox manifest
      A audioplayers
      A path_provider
      - BoringSSL
      - Firebase
      - FirebaseAnalytics
      - FirebaseAuth
      - FirebaseCore
      - FirebaseDatabase
      - FirebaseFirestore
      - FirebaseInstanceID
      - Flutter
      - GTMSessionFetcher
      - GoogleToolboxForMac
      - Protobuf
      - cloud_firestore
      - firebase_auth
      - gRPC
      - gRPC-Core
      - gRPC-ProtoRPC
      - gRPC-RxLibrary
      - leveldb-library
      - nanopb

    Downloading dependencies

    -> Using BoringSSL (9.0)

    -> Using Firebase (4.5.0)

    -> Using FirebaseAnalytics (4.0.4)

    -> Using FirebaseAuth (4.3.1)

    -> Using FirebaseCore (4.0.10)

    -> Using FirebaseDatabase (4.1.0)

    -> Using FirebaseFirestore (0.9.1)

    -> Using FirebaseInstanceID (2.0.5)

    -> Using Flutter (1.0.0)

    -> Using GTMSessionFetcher (1.1.12)

    -> Using GoogleToolboxForMac (2.1.3)

    -> Using Protobuf (3.4.0)

    -> Installing audioplayers (0.0.1)

    -> Using cloud_firestore (0.0.1)

    -> Using firebase_auth (0.0.1)

    -> Using gRPC (1.6.5)

    -> Using gRPC-Core (1.6.5)

    -> Using gRPC-ProtoRPC (1.6.5)

    -> Using gRPC-RxLibrary (1.6.5)

    -> Using leveldb-library (1.20)

    -> Using nanopb (0.3.8)

    -> Installing path_provider (0.0.1)
      - Running pre install hooks
    [!] Pods written in Swift can only be integrated as frameworks; add `use_frameworks!` to your Podfile or target to opt into using it. The Swift Pod being used is: audioplayers

    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/installer/xcode/target_validator.rb:115:in `block (2 levels) in verify_framework_usage'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/installer/xcode/target_validator.rb:110:in `each'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/installer/xcode/target_validator.rb:110:in `block in verify_framework_usage'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/installer/xcode/target_validator.rb:107:in `each'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/installer/xcode/target_validator.rb:107:in `verify_framework_usage'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/installer/xcode/target_validator.rb:38:in `validate!'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/installer.rb:405:in `validate_targets'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/installer.rb:118:in `install!'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/command/install.rb:41:in `run'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/claide-1.0.2/lib/claide/command.rb:334:in `run'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/lib/cocoapods/command.rb:52:in `run'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/gems/cocoapods-1.3.1/bin/pod:55:in `<top (required)>'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/bin/pod:23:in `load'
    /usr/local/Cellar/cocoapods/1.3.1/libexec/bin/pod:23:in `<main>'
Error output from CocoaPods:
↳

    [!] Automatically assigning platform ios with version 8.0 on target Runner because no platform was specified. Please specify a platform for this target in your Podfile. See `https://guides.cocoapods.org/syntax/podfile.html#platform`.
Error running pod install
Error launching application on iPhone X.

flutter --version gives me this output:
Flutter • channel master • https://github.com/flutter/flutter.git
Framework • revision<PHONE_NUMBER> (3 weeks ago) • 2017-11-01 22:31:29 -0700
Engine • revision 88cf09ffe6
Tools • Dart 1.25.0-dev.11.0

Am I missing anything?

Hi @andreugrimalt
Check the file ios/Podfile. It should have use_frameworks! as the first command of the target 'Runner' do block.
Like this:
target 'Runner' do
  use_frameworks!

  # Pods for Runner
  /// ...

Let's add that instruction to the README in a visible place.

@luanpotter I think I've found a better solution.

There was a unnecessary swift file forcing the calling of use_frameworks!.
I did some tests and it is not necessary anymore. The problem will be solved in the next release of both libs.
Regards

Thanks @feroult !

Nice, @feroult ! I'm closing this and the related audioplayers issue, if anything else arises, please let us know!


[Bug]: updating Erigon fails when updating all

Contact Details
No response
Describe the bug!
When pressing "UPDATE ALL" and Erigon got an update as well, Erigon's pruning config get's changed always to --prune=default which makes it break on startup when another pruning method was configured previously.
Attach a Screenshot!
No response
Expected Behaviour
Don't change pruning settings on "UPDATE ALL"
Help us reproduce your bug!
see above
Launcher Version
2.0.0-rc.22
Desktop OS
Mac
Node Server OS
Ubuntu 22.04
Affected Service
Erigon
Relevant log output
#150: not allowed change of --prune flag, last time you used: --prune.h.older=90000 --prune.t.older=90000 --prune.c.older=90000 --prune.r.before=11052984

#149: [EROR] [09-28|08:17:43.660] Erigon startup                           err="not allowed change of --prune flag, last time you used: --prune.h.older=90000 --prune.t.older=90000 --prune.c.older=90000 --prune.r.before=11052984"

Acknowledgment

[X] I looked through the existing bugs of this project

"fixed" with #1479 (removed old Update Scripts)


Add an additional confirmation message for deleting a student.

Is your feature request related to a problem? Please describe.
We have implemented a confirmation command for clearing the entire npc_track list, but this should also be extended to
deleting individual students. Otherwise should the user delete the wrong student on accident, they would need to go
through the painstaking process for adding all of their details again.
This is an issue to be rectified before v1.3 ends.

[X] I have added the relevant tags related to this feature request.

Describe the solution you'd like
A confirmation message to delete a student from the application, similar to what clear does.
Additional context
NIL

The key difference is that clear is a command that is only 5 letters with no parameters and is easily mistypable.
With delete, the user of the application will explicitly have to type 6 letters, as well as scrutinize the index of the person they want to delete.
I don't think there's strictly a need for this feature, what do you think?

We have decided that this was out of scope for v1.3, but we are open if you would like to make a pull request for this issue.


doc: change variable names in AtPrime.lean

This commit changes the names of the prime ideal to I to match the docstrings. It also fixes an instance of Localization.AtPrime taking too many arguments in a docstring.

Thanks!
bors merge


Fix misidentified init system

Add code to prepare.yml to install init on Debian images.  This is a workaround for the fact that testinfra currently misidentifies the init system on these hosts.
See philpep/testinfra#416 for more details.
I also took the opportunity to pull in upstream changes.

The fact that the (unneeded and unwanted) JAvaScript LGTM analysis fails is a red herring.  Please disregard.


rpmbuild: update version in spec file

Coverage remained the same at 79.973% when pulling 418e9c30d4d90f95e9dcdb3d3cbd46088ceb314e on rrojo/udpate-version into<PHONE_NUMBER>fa2305e87eb4287eb15a2efd281670 on master.

won't update version until we are making 1.4.1 release.


JSON output for batch processing is not a valid JSON

Currently, when pyradiomics script is used in the batch mode, and the input is more than one combination of image/mask, the output is a list of dictionaries, which is not a valid JSON document:
{
  "Image": "image1",
  "Mask": "mask1",
  ...
}
{
  "Image": "image2",
  "Mask": "mask2",
  ...
}

Should we revise this and instead output something like
{
  {
    "Image": "image1",
    "Mask": "mask1",
    ...
  },
  {
    "Image": "image2",
    "Mask": "mask2",
    ...
  }
}

@JoostJM what do you think?

@fedorov, I agree,
While investigating this, I also discovered that JSON does not serialize numpy arrays by default. Therefore, I also added a custom serializer to deal with this (by converting to python list).
This is part of PR #481, commit 7f9e67c


Out-of-date examples

This library is by far the most comprehensive deep learning framework in C, and is strangely underrated.
One possible reason is that the examples have not been updated for the new versions, and thus, newcomers cannot appreciate the performance of this library.
function itoa has been changed to itoa_n.
function fully_connected needs mode as the 11th argument.

yes i have to update the tests according to the new version, thanks for the opened issue, the update will be done asap

The tests available are now:
test1/
test2/
test3/
test4/
test5/
test6/
test7/
test9/
test10/
test11/
test12/
test13/
test16/
test17/
test18/
The transformer test must need further tuning cause the decoder is not implemented correctly, it will be in the near future.
Closing the Issue

Thanks for the update. I suggest to include more basic examples based on numeric data read from a plain data file, as people can easily manipulate the data or import other databases to compare the performance of this library with others.

i will try to change the format of the cifar-10 dataset i use for the tests and maybe add other examples

I tried to modify test1 by replacing the CIFAR-10 input with a simple set of input/output (sine and its divided value).
int main(){

    srand(time(NULL));
    // Initializing Training resources
    int i,j,k,z,training_instances = 500,input_dimension = 1,output_dimension = 1, middle_neurons = 3;
    int n_layers = 2;
    int batch_size = 30,threads = 30;
    int epochs = 5000;
    unsigned long long int t = 1;
    int size = 0;
    char temp[2];
    temp[1] = '\0';
    float** errors = (float**)malloc(sizeof(float*)*batch_size);

    for(i = 0; i < batch_size; i++){
        errors[i] = (float*)calloc(output_dimension,sizeof(float));
    }
    // Model Architecture
    fcl** fcls = (fcl**)malloc(sizeof(fcl*)*2);
    fcls[0] = fully_connected(input_dimension,middle_neurons,0,NO_DROPOUT,TANH,0,0,NO_NORMALIZATION,GRADIENT_DESCENT,FULLY_FEED_FORWARD, STANDARD);
    fcls[1] = fully_connected(middle_neurons,output_dimension,1,NO_DROPOUT,TANH,0,0,NO_NORMALIZATION,GRADIENT_DESCENT,FULLY_FEED_FORWARD, STANDARD);
    model* m = network(n_layers,0,0,2,NULL,NULL,fcls);
    model** batch_m = (model**)malloc(sizeof(model*)*batch_size);
    float** ret_err = (float**)malloc(sizeof(float*)*batch_size);
    for(i = 0; i < batch_size; i++){
        batch_m[i] = copy_model(m);
    }
    int ws = count_weights(m);
    float lr = 0.7, momentum = 0.9, lambda = 1;
    // Reading the data in a char** vector
    float** inputs = (float**)malloc(sizeof(float*)*training_instances);
    float** outputs = (float**)malloc(sizeof(float*)*training_instances);
    // Creating an input of sine and output of input divided by 2
    for(i = 0; i < training_instances; i++){
        inputs[i] = (float*)malloc(sizeof(float)*input_dimension);
        outputs[i] = (float*)calloc(output_dimension,sizeof(float));
        for(j = 0; j < input_dimension; j++){inputs[i][j] = sin(PI*i*(j+1)*0.01);}
        for(j = 0; j < output_dimension; j++){outputs[i][j] = inputs[i][j]/2;}
    }

    printf("Training phase!\n");
    save_model(m,0);
    // Training
    for(k = 0; k < epochs; k++){
        if(k == 10)
            lr = 0.0001;
        else if(k == 15)
            lr = 0.00005;
        printf("Starting epoch %d/%d\n",k+1,epochs);
        // Shuffling before each epoch
        shuffle_float_matrices(inputs,outputs,training_instances);
        for(i = 0; i < training_instances/batch_size; i++){
            //printf("Mini batch number: %d\n",i+1);
            // Feed forward and backpropagation
            model_tensor_input_ff_multicore(batch_m,input_dimension,1,1,&inputs[i*batch_size],batch_size,threads);
            for(j = 0; j < batch_size; j++){
                derivative_focal_loss_array(batch_m[j]->fcls[1]->post_activation,outputs[i*batch_size+j],errors[j],2,output_dimension);
            }
            model_tensor_input_bp_multicore(batch_m,input_dimension,1,1,&inputs[i*batch_size],batch_size,threads,errors,output_dimension,ret_err);
            // sum the partial derivatives in m obtained from backpropagation
            for(j = 0; j < batch_size; j++){
                sum_model_partial_derivatives(batch_m[j],m,m);
            }
            // update m, reset m and copy the new weights in each instance of m of the batch
            update_model(m,lr,momentum,batch_size,NESTEROV,NULL,NULL,NO_REGULARIZATION,ws,lambda,&t);
            reset_model(m);
            for(j = 0; j < batch_size; j++){
                paste_model(m,batch_m[j]);
                reset_model(batch_m[j]);
            }

        }
        // Saving the model
        save_model(m,k+1);
    }

    // Deallocating Training resources
    free_model(m);
    for(i = 0; i < batch_size; i++){
        free_model(batch_m[i]);
        free(errors[i]);
    }
    free(errors);
    free(batch_m);
    free(ret_err);

    // Initializing Testing resources
    model* test_m;
    int size2 = 0;
    int testing_instances = 100;
    char temp2[256];

    long long unsigned int** cm;

    printf("Testing phase!\n");
    double error = 0;
    // Testing
    for(k = 0; k < epochs+1; k++){
        printf("Model N. %d/%d\n",k+1,epochs);
        // Loading the model
        char temp3[5];
        temp3[0] = '.';
        temp3[1] = 'b';
        temp3[2] = 'i';
        temp3[3] = 'n';
        temp3[4] = '\0';
        itoa_n(k,temp2);
        strcat(temp2,temp3);
        test_m = load_model(temp2);
        for(i = 0; i < testing_instances; i++){
            // Feed forward

            model_tensor_input_ff(test_m,input_dimension,1,1,inputs[i]);
            for(j = 0; j < output_dimension; j++){
                error+=focal_loss(test_m->fcls[1]->post_activation[j],outputs[i][j],2);
            }
            // Printing the output of each neuron including the last layer
            for(j = 0; j < 1; j++){printf("%f %f -> %f %f -> %f %f\n",inputs[i][j],outputs[i][j],test_m->fcls[0]->pre_activation[j],test_m->fcls[0]->post_activation[j],test_m->fcls[1]->pre_activation[j],test_m->fcls[1]->post_activation[j]);}
            if(!i)
                cm = confusion_matrix(test_m->fcls[1]->post_activation, outputs[i],NULL, 10,0.5);
            else
                cm = confusion_matrix(test_m->fcls[1]->post_activation, outputs[i],cm, 10,0.5);
            reset_model(test_m);
        }
        printf("Error: %lf\n",error);
        printf("Accuracy, Precision, Sensitivity, Specificity:\n");
        print_accuracy(cm,output_dimension);
        print_precision(cm,output_dimension);
        print_sensitivity(cm,output_dimension);
        print_specificity(cm,output_dimension);
        for(i = 0; i < output_dimension*2; i++){
            free(cm[i]);
        }
        free(cm);
        error = 0;
        free_model(test_m);
    }
    // Deallocating testing resources

    free(inputs);
    free(outputs);
}

but I was unable to make it work.

yes because the error is a focal loss and you used the tanh as final activation function for the last layer tanh is in [-1,1] considering that the focal loss uses the log, a log of a negative value would have produced some nan for sure.
try using the MSE loss. However test1 is a little bit outdated.
With new apis you can get a faster training and less memory usage.
For your purpose you can try this code:
#include <llab.h>

int main(){
    srand(time(NULL));
    // Initializing Training resources
    int i,j,k,z,training_instances = 50000,input_dimension = 784,output_dimension = 10, middle_neurons = 100,counter;
    float lh_a = 0.5;
    int lh_k = 5;
    int n_layers = 2;
    int batch_size = 10,threads = 8;
    int epochs = 5;
    unsigned long long int t = 1;
    char** ksource = (char**)malloc(sizeof(char*));
    char* filename = "../data/train.bin";
    int size = 0;
    char temp[2];
    float b1 = BETA1_ADAM;
    float b2 = BETA2_ADAM;
    temp[1] = '\0';
    float** errors = (float**)malloc(sizeof(float*)*batch_size);

    for(i = 0; i < batch_size; i++){
        errors[i] = (float*)calloc(output_dimension,sizeof(float));
    }
    // Model Architecture
    fcl** fcls = (fcl**)malloc(sizeof(fcl*)*2);
    fcls[0] = fully_connected(input_dimension,middle_neurons,0,NO_DROPOUT,SIGMOID,0,0,NO_NORMALIZATION,GRADIENT_DESCENT,FULLY_FEED_FORWARD, STANDARD);
    fcls[1] = fully_connected(middle_neurons,output_dimension,1,NO_DROPOUT,TANH,0,0,NO_NORMALIZATION,GRADIENT_DESCENT,FULLY_FEED_FORWARD, STANDARD);
    model* m = network(n_layers,0,0,2,NULL,NULL,fcls);
    set_model_error(m,MSE_LOSS,0,0,0,NULL,output_dimension);

    model** batch_m = (model**)malloc(sizeof(model*)*batch_size);
    float** ret_err = (float**)malloc(sizeof(float*)*batch_size);
    for(i = 0; i < batch_size; i++){
        batch_m[i] = copy_model_without_learning_parameters(m);
    }
    int ws = count_weights(m);
    float lr = 0.001, momentum = 0.9, lambda = 0.0001;

    // Reading the data in a char** vector
    //read_file_in_char_vector(ksource,filename,&size);
    float** inputs = (float**)malloc(sizeof(float*)*training_instances);
    float** outputs = (float**)malloc(sizeof(float*)*training_instances);
    // Putting the data in float** vectors
    /*for(i = 0; i < training_instances; i++){
        inputs[i] = (float*)malloc(sizeof(float)*input_dimension);
        outputs[i] = (float*)calloc(output_dimension,sizeof(float));
        for(j = 0; j < input_dimension+1; j++){
            temp[0] = ksource[0][i*(input_dimension+1)+j];
            if(j == input_dimension)
                outputs[i][atoi(temp)] = 1;
            else
                inputs[i][j] = atof(temp);
        }
    }*/
    for(i = 0; i < training_instances; i++){
        inputs[i] = (float*)malloc(sizeof(float)*input_dimension);
        outputs[i] = (float*)calloc(output_dimension,sizeof(float));
        for(j = 0; j < input_dimension; j++){inputs[i][j] = sin(PI*i*(j+1)*0.01);}
        for(j = 0; j < output_dimension; j++){outputs[i][j] = inputs[i][j]/2;}
    }

    printf("Training phase!\n");

    counter = 0;
    save_model(m,0);
    // Training

    for(k = 0; k < epochs; k++){
        if(k == 4)
            lr = 0.0001;
        else if(k == 5)
            lr = 0.00005;
        printf("Starting epoch %d/%d\n",k+1,epochs);
        // Shuffling before each epoch
        shuffle_float_matrices(inputs,outputs,training_instances);
        for(i = 0; i < training_instances/batch_size; i++){
            printf("Mini batch number: %d\n",i+1);
            // Feed forward and backpropagation
            ff_error_bp_model_multicore_opt(batch_m,m,input_dimension,1,1,&inputs[i*batch_size],batch_size,batch_size,&outputs[i*batch_size],ret_err);
            // sum the partial derivatives in m obtained from backpropagation
            sum_models_partial_derivatives(m,batch_m,batch_size);
            // update m, reset m and copy the new weights in each instance of m of the batch
            update_model(m,lr,momentum,batch_size,ADAM,&b1,&b2,L2_REGULARIZATION,ws,lambda,&t);
            reset_model(m);
            for(j = 0; j < batch_size; j++){
            	reset_model_without_learning_parameters(batch_m[j]);
            }

            update_training_parameters(&b1,&b2,&t,m->beta1_adam,m->beta2_adam);

        }
        // Saving the model
        save_model(m,k+1);
    }

    // Deallocating Training resources
    free(ksource[0]);
    free(ksource);
    free_model(m);

    for(i = 0; i < batch_size; i++){
        free_model_without_learning_parameters(batch_m[i]);
        free(errors[i]);
    }
    free(errors);
    free(batch_m);
    free(ret_err);
    for(i = 0; i < training_instances; i++){
        free(inputs[i]);
        free(outputs[i]);
    }
    free(inputs);
    free(outputs);

    // Initializing Testing resources
    model* test_m;
    char** ksource2 = (char**)malloc(sizeof(char*));
    char* filename2 = "../data/test.bin";
    int size2 = 0;
    int testing_instances = 50000;
    char temp2[256];
    //read_file_in_char_vector(ksource2,filename2,&size);
    float** inputs_test = (float**)malloc(sizeof(float*)*testing_instances);
    float** outputs_test = (float**)malloc(sizeof(float*)*testing_instances);
    // Putting the data in float** vectors
    for(i = 0; i < training_instances; i++){
        inputs_test[i] = (float*)malloc(sizeof(float)*input_dimension);
        outputs_test[i] = (float*)calloc(output_dimension,sizeof(float));
        for(j = 0; j < input_dimension; j++){inputs_test[i][j] = sin(PI*i*(j+1)*0.01);}
        for(j = 0; j < output_dimension; j++){outputs_test[i][j] = inputs_test[i][j]/2;}
    }
    //long long unsigned int** cm;

    printf("Testing phase!\n");
    double error = 0;
    // Testing
    for(k = 0; k < epochs+1; k++){
        printf("Model N. %d/%d\n",k+1,epochs);
        // Loading the model
        char temp3[5];
        temp3[0] = '.';
        temp3[1] = 'b';
        temp3[2] = 'i';
        temp3[3] = 'n';
        temp3[4] = '\0';
        itoa_n(k,temp2);
        strcat(temp2,temp3);
        test_m = load_model(temp2);
        for(i = 0; i < testing_instances; i++){
            // Feed forward

            model_tensor_input_ff(test_m,input_dimension,1,1,inputs_test[i]);
            for(j = 0; j < output_dimension; j++){
                error+=mse(test_m->fcls[1]->post_activation[j],outputs_test[i][j]);
            }
            /*
            if(!i)
                cm = confusion_matrix(test_m->fcls[1]->post_activation, outputs_test[i],NULL, 10,0.5);
            else
                cm = confusion_matrix(test_m->fcls[1]->post_activation, outputs_test[i],cm, 10,0.5);
            */
            reset_model(test_m);
        }
        printf("Error: %lf\n",error);
        /*printf("Accuracy, Precision, Sensitivity, Specificity:\n");
        print_accuracy(cm,output_dimension);
        print_precision(cm,output_dimension);
        print_sensitivity(cm,output_dimension);
        print_specificity(cm,output_dimension);
        for(i = 0; i < output_dimension*2; i++){
            free(cm[i]);
        }
        free(cm);*/
        error = 0;
        free_model(test_m);
    }
}

As you can see i added the error with this line:
set_model_error(m,MSE_LOSS,0,0,0,NULL,output_dimension);
Also i didn't manually computed the derivatives, the function:
ff_error_bp_model_multicore_opt(batch_m,m,input_dimension,1,1,&inputs[i*batch_size],batch_size,batch_size,&outputs[i*batch_size],ret_err);
takes as many models without learning parameters (without weights and biases but only with buffers to store some outputs and partial derivatives) as the number of the size of the mini batch.
For each one it computes the feedforward, the loss (According to the loss set manually with the previous function) backpropagates the error (For each models it starts a new thread, this function must be updated to make it more efficient).
With this code i did what you wanted to do with the sin.
This is the errors it showed epoch by epoch:
Model N. 1/5
Error: 81192.210139
Model N. 2/5
Error: 95.010154
Model N. 3/5
Error: 21.562515
Model N. 4/5
Error: 14.446248
Model N. 5/5
Error: 9.016324
Model N. 6/5
Error: 0.253015

As you can see the error decreases starting from 81192... and reaching 0.2 after 5 epochs

Also, if you want a general model without everytime compiling your code you can look at the parsing file and take some examples from the python version of llab:
https://github.com/ez-spark/Pyllab/tree/main/test/model
Here there are some examples where the models are set on a .txt file, the C code can parse these kind of file and create a model starting from the configuration you have set on your .txt file.
The function that parses these kind of file is
model* parse_model_file(char* filename);
that you can find in parser.c.
If you have some doubts about how your configuration files should look like to use such a function do not hesitate to write me here :)

WOW! your new example worked charmingly. Thanks for that.
I am an avid C enthusiast. Thus, I prefer to stick with this library rather than its Python version. This is indeed why I came here. I used FANN for a long time, but it is quite outdated, with no plan for the inclusion of trending techniques such as RNN and CNN (not to mention Transformer).
Your library is quite impressive. When I found it, I couldn't believe it was real (such a comprehensive framework, but not popular). It is still strange to me.
I am more into statistical analysis of time series and use deep learning as a side inspiration. Therefore, I am not a capable person to contribute to this well-established library. However, if you don't mind, I can post my attempts (like the above) to gain your corrections. This probably attracts the attention of other people here or via search engines to the powerful capabilities of this library.
Whatever you do, please do not give up. This is currently the most advanced deep learning library in C, and we need to have such a comprehensive library in C. Thank you for all your efforts.

Just a quick question. What is your architecture suggestion for a non-linear dependency of input/output? For example:
for(j = 0; j < input_dimension; j++){inputs[i][j] = sin(PI*i*(j+1)*0.01);}
for(j = 0; j < output_dimension; j++){outputs[i][j] = sin(PI*i*(j+1)*0.022);}

Just use fully-connected layers with 2+ layers. Tanh as final activation function is ok. Maybe for the loss you can use the Huber loss or the MSE. take into account that llab offers also convolutional and transposed convolutional layers and pooling, padding RNN, and so on... but for this purpose they are pretty useless. For the hidden layers i would use RELU or LEAKY_RELU
Also maybe i would add some normalization layers inside the fully connected layers. It has been proved that normalization usually increases the accuracy.
During training however is usually used batch normalization rather than layer normalization so if you want to add normalization i would go for batch normalization. The problem is that for batch normalization there are the APIs but you must set them manually and it s a little bit of work because you need to split the network and is not ideal for coding (in the wiki page for what i remember there is an example of batch normalization implementation).
BUT there is an alternative to batch normalization taken from a paper published something like 2-3 years ago that is 'adaptive gradient clipping' that shows same performance of batch normalization and is really simple to implement because you have just to call a function after each iteration and is less computational costly than batch normalization.
The only issue that i witnessed with adaptive gradient clipping is that after a while it easily overfits the data.
So i would suggest to add DROPOUT to some of your fully connected layers with a dropout threshold of 0.2 or 0.3 to generalize better.
You can see the adaptive gradient clipping example in the test16 that calls the function:
adaptive_gradient_clipping_model(m,0.01,1e-3);
If you used DROPOUT you need during inference time to set the layers to DROPOUT_TEST (aka DROPOUT INFERENCE) and set the dropout threshold to 1-dropout_threshold_during_training.
Back to your problem, if your problem is just something that simple as the problem of the sin you showed me, just use 3 fully connected layers the middle one with dropout and that's it. No need to all this fancy stuff i explainedto you.
If you have a more complex problem and you try to create a deep architecture remember to add the residual layers because they add the input of the previous layer to the output of some other layers. This is needed in very deep network because of the vanishing gradient problem

Many thanks for the suggestions and explanations. Quite informative. I followed your advice in designing the model architecture but had no success. In fact, I believe it is not possible to capture such non-linear dependency by feed forward algorithms. I came here in search of RNN and LSTM.
tests 16-18 are still outdated, and I am trying to modify them to work with the current version.
Surely, GPU implementation is a must for such an extensive library, but not my need. I have enough CPU cores and work with one-dimensional datasets. Therefore, multithreading is enough to handle my workload, as opposed to common datasets of images with convolutional layers.
Your library has everything I need. I just need to figure out the overall structure of the programme. It is quite different from FANN, which I am familiar with. On the other hand, it is quite comprehensive, and it is not quick and easy to walk through functions, but I am trying! :)

If you open a repo with your data on your account i can help you using llab maybe trying different configurations and making pull requests and then you can accept or not.

Thanks for kindly offering help. I have not yet moved to a real dataset. At this stage, I just try to find a model architecture that can find the relationship between two sine waves:
input = sin(x * 0.01);
output = sin(x * 0.02);

If I find a suitable model to handle this elementary relationship, I can then plan for more complicated real-world cases.
Thank you again for your supportive attention.

Mh, so i tried to create a sort of autoencoder, but the results are not that promising. I have just run 3 epochs but it seems that after a while it stucks in a local oprima, The code:
`#include <llab.h>
int main(){
srand(time(NULL));
// Initializing The neede variables
int i,j,k,z,training_instances = 80000,input_dimension = 1,output_dimension = 1, first_middle_neurons = 1000,second_middle_neurons = 200, third_middle_neurons = 1000;
int n_layers = 6;
int batch_size = 20,threads = 20;
int epochs = 20;
unsigned long long int t = 1;
float b1 = BETA1_ADAM;
float b2 = BETA2_ADAM;
// Model Architecture
fcl** fcls = (fcl**)malloc(sizeof(fcl*)*4);
cl** cls = (cl**)malloc(sizeof(cl*)*2);
rl** rls = (rl**)malloc(sizeof(rl*));
fcls[0] = fully_connected(input_dimension,first_middle_neurons,0,NO_DROPOUT,RELU,0,0,NO_NORMALIZATION,GRADIENT_DESCENT,FULLY_FEED_FORWARD, STANDARD);
fcls[1] = fully_connected(first_middle_neurons,second_middle_neurons,2,DROPOUT,RELU,0.3,0,NO_NORMALIZATION,GRADIENT_DESCENT,FULLY_FEED_FORWARD, STANDARD);
fcls[2] = fully_connected(second_middle_neurons,third_middle_neurons,3,DROPOUT,RELU,0.3,0,NO_NORMALIZATION,GRADIENT_DESCENT,FULLY_FEED_FORWARD, STANDARD);
fcls[3] = fully_connected(third_middle_neurons,output_dimension,5,NO_DROPOUT,TANH,0,0,NO_NORMALIZATION,GRADIENT_DESCENT,FULLY_FEED_FORWARD, STANDARD);
cls[0] = convolutional(1, 1, first_middle_neurons, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, NO_NORMALIZATION, NO_ACTIVATION, MAX_POOLING, 0, NO_CONVOLUTION,FREEZE_TRAINING, FULLY_FEED_FORWARD, 1);
cls[1] = convolutional(1, 1, first_middle_neurons, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, NO_NORMALIZATION, NO_ACTIVATION, MAX_POOLING, 0, NO_CONVOLUTION,FREEZE_TRAINING, FULLY_FEED_FORWARD, 4);
rls[0] = residual(1,1,first_middle_neurons, 2, cls);
model* m = network(n_layers,1,0,4,rls,NULL,fcls);
// set the model error
set_model_error(m,MSE_LOSS,0,0,0,NULL,output_dimension);
// using copied models for multi threading
model** batch_m = (model**)malloc(sizeof(model*)*batch_size);
for(i = 0; i < batch_size; i++){
    batch_m[i] = copy_model_without_learning_parameters(m);
}
// some hyper parameters
float lr = 0.001, momentum = 0.9, decay = lr/epochs;

// instances for training
float** inputs = (float**)malloc(sizeof(float*)*training_instances);
float** outputs = (float**)malloc(sizeof(float*)*training_instances);
for(i = 0; i < training_instances; i++){
    inputs[i] = (float*)malloc(sizeof(float)*input_dimension);
    outputs[i] = (float*)calloc(output_dimension,sizeof(float));
    float r = r2()*2*PI;//range in [0,2pi]
    for(j = 0; j < input_dimension; j++){inputs[i][j] = sin(r);}
    for(j = 0; j < output_dimension; j++){outputs[i][j] = sin(r*2);}
}
// instances for validation
float** inputs_val = (float**)malloc(sizeof(float*)*training_instances/2);
float** outputs_val = (float**)malloc(sizeof(float*)*training_instances/2);
for(i = 0; i < training_instances/2; i++){
    inputs_val[i] = (float*)malloc(sizeof(float)*input_dimension);
    outputs_val[i] = (float*)calloc(output_dimension,sizeof(float));
    float r = r2()*2*PI;//range in [0,2pi]
    for(j = 0; j < input_dimension; j++){inputs_val[i][j] = sin(r);}
    for(j = 0; j < output_dimension; j++){outputs_val[i][j] = sin(r*2);}
}

printf("Training phase!\n");

save_model(m,0);
// Training
double error = 0, last_error = -1;
for(k = 0; k < epochs; k++){
    printf("Validation on validation set\n");
    // Saving the model
    // using the validation dataset to stop the training if it overfits the data
    model* val_m = copy_model(m);
    make_the_model_only_for_ff(val_m);
    for(i = 0; i < training_instances/2; i++){
        model_tensor_input_ff(val_m,input_dimension,1,1,inputs_val[i]);
        for(j = 0; j < output_dimension; j++){
            error+=mse(get_output_layer_from_model(val_m)[j],outputs_val[i][j]);
        }
        reset_model(val_m);
    }
    free_model(val_m);
    printf("Error: %lf\n",error);
    if(last_error >= 0)
        if (error > last_error)
            break;
    last_error = error;
    error = 0;

    printf("Validation on training set\n");
    // Saving the model
    // using the validation dataset to stop the training if it overfits the data
    val_m = copy_model(m);
    make_the_model_only_for_ff(val_m);
    for(i = 0; i < training_instances; i++){
        model_tensor_input_ff(val_m,input_dimension,1,1,inputs[i]);
        for(j = 0; j < output_dimension; j++){
            error+=mse(get_output_layer_from_model(val_m)[j],outputs[i][j]);
        }
        reset_model(val_m);
    }
    free_model(val_m);
    printf("Error: %lf\n",error);
    last_error = error;
    error = 0;

    printf("Starting epoch %d/%d\n",k+1,epochs+1);
    // Shuffling before each epoch
    shuffle_float_matrices(inputs,outputs,training_instances);
    for(i = 0; i < training_instances/batch_size; i++){
        if(!(i%100))
            printf("batch number: %d/%d\n",i,training_instances/batch_size);
        // Feed forward and backpropagation
        ff_error_bp_model_multicore_opt(batch_m,m,input_dimension,1,1,&inputs[i*batch_size],batch_size,batch_size,&outputs[i*batch_size],NULL);
        // sum the partial derivatives in m obtained from backpropagation
        sum_models_partial_derivatives(m,batch_m,batch_size);
        // update m, reset m and copy the new weights in each instance of m of the batch
        update_model(m,lr,momentum,batch_size,ADAM,&b1,&b2,NO_REGULARIZATION,0,0,&t);
        reset_model(m);
        for(j = 0; j < batch_size; j++){
            reset_model_without_learning_parameters(batch_m[j]);
        }
        adaptive_gradient_clipping_model(m,0.01,1e-3);
        update_training_parameters(&b1,&b2,&t,m->beta1_adam,m->beta2_adam);
        update_lr(&lr, 0.0001, 0.01,0.001, decay, k+1, epochs, LR_TIME_BASED_DECAY);
    }

    // Saving the model
    save_model(m,k+1);
}

// Deallocating Training resources
free_model(m);
for(i = 0; i < batch_size; i++){
    free_model_without_learning_parameters(batch_m[i]);
}
free(batch_m);
free_matrix((void**)inputs,training_instances);
free_matrix((void**)outputs,training_instances);

// using the inputs_val and outputs_val to test

for(i = 0; i < training_instances/2; i++){
    float r = r2()*2*PI;//range in [0,2pi]
    for(j = 0; j < input_dimension; j++){inputs_val[i][j] = sin(r);}
    for(j = 0; j < output_dimension; j++){outputs_val[i][j] = sin(r*2);}
}
printf("Testing phase!\n");

char temp2[256];
error = 0;
// Testing
for(k = 0; k < epochs+1; k++){
    printf("Model N. %d/%d\n",k+1,epochs+1);
    // Loading the model
    char temp3[5];temp3[0] = '.';temp3[1] = 'b';temp3[2] = 'i';temp3[3] = 'n';temp3[4] = '\0';
    itoa_n(k,temp2);
    strcat(temp2,temp3);
    m = load_model(temp2);
    make_the_model_only_for_ff(m);
    for(i = 0; i < training_instances/2; i++){
        // Feed forward

        model_tensor_input_ff(m,input_dimension,1,1,inputs_val[i]);
        for(j = 0; j < output_dimension; j++){
            error+=mse(get_output_layer_from_model(m)[j],outputs_val[i][j]);
        }
        reset_model(m);
    }
    printf("Error: %lf\n",error);
    error = 0;
    free_model(m);
}
free_matrix((void**)inputs,training_instances);
free_matrix((void**)outputs,training_instances);

}
The results after 2 epochs:Training phase!
Validation on validation set
Error: 15334.032870
Validation on training set
Error: 30814.696431
Starting epoch 1/21
batch number: 0/4000
batch number: 100/4000
batch number: 200/4000
batch number: 300/4000
batch number: 400/4000
batch number: 500/4000
batch number: 600/4000
batch number: 700/4000
batch number: 800/4000
batch number: 900/4000
batch number: 1000/4000
batch number: 1100/4000
batch number: 1200/4000
batch number: 1300/4000
batch number: 1400/4000
batch number: 1500/4000
batch number: 1600/4000
batch number: 1700/4000
batch number: 1800/4000
batch number: 1900/4000
batch number: 2000/4000
batch number: 2100/4000
batch number: 2200/4000
batch number: 2300/4000
batch number: 2400/4000
batch number: 2500/4000
batch number: 2600/4000
batch number: 2700/4000
batch number: 2800/4000
batch number: 2900/4000
batch number: 3000/4000
batch number: 3100/4000
batch number: 3200/4000
batch number: 3300/4000
batch number: 3400/4000
batch number: 3500/4000
batch number: 3600/4000
batch number: 3700/4000
batch number: 3800/4000
batch number: 3900/4000
Validation on validation set
Error: 10815.763572
Validation on training set
Error: 21687.032894
Starting epoch 2/21
batch number: 0/4000
batch number: 100/4000
batch number: 200/4000
batch number: 300/4000
batch number: 400/4000
`
It clearly decrease on training and valiadation set but the error keeps being stuck on 10k for the validation even with different configurations and so on. It could be everything, maybe just increasing the training data, or the batch size, or using different optimizations instead of ADAM or having less learning rate. I mean it could be really everything

This is the error on validation and training after 2 epochs with leaky_relu instead or relu:
Training phase!
Validation on validation set
Error: 19858.760166
Validation on training set
Error: 39628.707041
Starting epoch 1/21
batch number: 0/4000
batch number: 100/4000
batch number: 200/4000
batch number: 300/4000
batch number: 400/4000
batch number: 500/4000
batch number: 600/4000
batch number: 700/4000
batch number: 800/4000
batch number: 900/4000
batch number: 1000/4000
batch number: 1100/4000
batch number: 1200/4000
batch number: 1300/4000
batch number: 1400/4000
batch number: 1500/4000
batch number: 1600/4000
batch number: 1700/4000
batch number: 1800/4000
batch number: 1900/4000
batch number: 2000/4000
batch number: 2100/4000
batch number: 2200/4000
batch number: 2300/4000
batch number: 2400/4000
batch number: 2500/4000
batch number: 2600/4000
batch number: 2700/4000
batch number: 2800/4000
batch number: 2900/4000
batch number: 3000/4000
batch number: 3100/4000
batch number: 3200/4000
batch number: 3300/4000
batch number: 3400/4000
batch number: 3500/4000
batch number: 3600/4000
batch number: 3700/4000
batch number: 3800/4000
batch number: 3900/4000
Validation on validation set
Error: 10515.439924
Validation on training set
Error: 20992.770086
Starting epoch 2/21
batch number: 0/4000
batch number: 100/4000
batch number: 200/4000
batch number: 300/4000
batch number: 400/4000
batch number: 500/4000
batch number: 600/4000
batch number: 700/4000
batch number: 800/4000
batch number: 900/4000
batch number: 1000/4000

Again, i tried with huber loss instead of MSE and it seems that keeps going to about 10k. I think that is global optimum for the network architecture i showed you. I think that if you create deeper architectures you can achieve lower loss easily.

Many thanks for your efforts. I will start experimenting with the model architecture you suggested.
It puzzled me for a while why it is so difficult for a neural network to understand a simple phase difference between two sine waves. I read some scholarly papers on the matter (for example, this one blaming the activation function). However, I still do not fathom what the underlying challenge is. And if it is really challenging, why has little attention been paid to this problem? Surely, there should be many applications depending on it, similar to the importance of FFT in understanding periodic series.

Mh maybe this is the problem. They blame relu and all relu variants as well as tanh to not being able to learn a periodic function. And that exactly what we were using xD. I could try to add their Snake_{a} function as activation function and see the results.
However there is something weird during inference time for the architecture i proposed to you.
During test i basically i plotted the sin(2x) where x starts from 0 and increases everytime by 0.01 and i plotted at the same time the output of the model.
It seems that, while the sin(2x) behaves as a sin in its periodic manner, the output of the model always gives random number really close to 0. However if you increase the output of the model multiplying it by a factor of 100 for example and plot the results it shows a sort of periodic function that doesnt seem a sin at all. It seems that it grasped that the function it tries to map is periodic but is not able to grasp the behaviour of its periodicity.
So maybe the authors of the papers are right

Maybe you can try a RNN using a Many2Many architecture.
Lets assume you use 20 lstm cells. as input you gives sin(x), sin(x+0.01), sin(x+0.02), ..., sin(x+0.19)
and as output sin(2x), sin(2(x+0.01)), ...

I implemented Snake and other activation functions in FANN but had no success. I am convinced feed forward cannot identify periodic dependency (or at least not with a straightforward model architecture). Since it was the end of FANN options, I looked for an alternative to implement RNN, particularly LSTM. I will start experimenting with RNN, but I need to get myself families with the structure of your library and underlying functions.
One possible explanation for the peculiar behaviour you observed is the periodic overlap of two sine waves (input and output). Unless reaching an error close to zero, the learning success is not reliable since the model could have identified the relationship between the input and the output at part of the dataset where both waves are moving in the same direction. If we feed only half of the cycle, even a feed forward model can identify the relationship of both curvatures. The problem arises when two waves move in opposite directions.

any improvements? did you try with RNNs?


Allow aql.transform to receive sql filepath (fix breaking change)

Fix aql.transform breaking change (sql as a file path when using the operator as a task).
Astronomer's Data team raised this issue in slack: https://astronomer.slack.com/archives/C02B.8SPT93K/p1680202480707669
Included in this PR:

Test case which illustrates data team's usage of aql.transform, which raises an exception in 1.5.3 (and used to work in 1.3.3)
Bug fix allows users to continue using the Python SDK as they used with Python SDK 1.3.3.

Open for discussion:
Do we want to continue supporting this use case? The Data team would like a task which allows them to transform both from a sql string and a file.sql. At the moment, transform_file should address this. Since we have introduced a breaking change, there is an argument for us to fix this.
Additional note:
The two broken tests are not related to the changes in this PR.

@tatiana IMO, we should be backward compatible and should only introduce a breaking changes in 2.0.


Strongly Encapsulate JDK Internals PolymorphicThrowableSchema

Java 17 Strongly Encapsulate JDK Internals, the code not working
public abstract class PolymorphicThrowableSchema extends PolymorphicSchema
{
static final java.lang.reflect.Field __cause;

static
{
    java.lang.reflect.Field cause;
    try
    {
        cause = Throwable.class.getDeclaredField("cause");
        cause.setAccessible(true);
    }
    catch (Exception e)
    {
        cause = null;
    }
    __cause = cause;
}

}

@snsyzf thanks for reporting.  Can you try testing against 1.8.0-SNAPSHOT and see if you still get problems?


The newest version 0.2.4 doesn't support Mathematica 11.3

Though I know this isn't a problem. But since MMA 12.1's front end is so slow. I have to go back to MMA 11.3. As a result , I can't use it.

I just add a new branch trying to add support for v11.3. You can remove the current package and install this new one:
PacletInstall["https://github.com/njpipeorgan/MathCompile/releases/download/v0.2.4/MathCompile-0.2.4.paclet"]

The problem of supporting versions prior to 12.0 is that NumericArray cannot be used for arguments and return values, but other features should work fine.

OK... I know the problem. At that time, I am using MinGW 32bit, not w64, so mma didn't recognize it.

OK... I know the problem. At that time, I am using MinGW 32bit, not w64, so mma didn't recognize it.

I see, thanks for letting me know.
Brian
On Tue, Jan 19, 2021 at 09:47 wuyudi<EMAIL_ADDRESS>wrote:

OK... I know the problem. At that time, I was using MinGW 32bit, not
w64, so mma didn't recognize it.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/njpipeorgan/MathCompile/issues/3#issuecomment-762888069,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC66HXKR2NNQEUNJMPOHI53S2WLQFANCNFSM4MLGZH4Q
.

I see, thanks for letting me know.
Brian
On Tue, Jan 19, 2021 at 09:47 wuyudi<EMAIL_ADDRESS>wrote:

OK... I know the problem. At that time, I was using MinGW 32bit, not
w64, so mma didn't recognize it.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/njpipeorgan/MathCompile/issues/3#issuecomment-762888069,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AC66HXKR2NNQEUNJMPOHI53S2WLQFANCNFSM4MLGZH4Q
.

Just some additional information.
It seems that the main problem is
https://reference.wolfram.com/language/ref/Typed.html?q=Typed
Typed is in 12.x or newer.
in the old version, RawArray can be used to replace NumericArray.
https://zhuanlan.zhihu.com/p/350280934


Bug: Error Handling for Nonexistent GitHub Usernames

Description:
When attempting to fetch data for a GitHub user, if the provided username does not exist, the application currently fails to display an appropriate error message. This results in confusion and frustration for users who encounter this issue, as they are left unsure why the data is not being retrieved.
Expected Behavior:
When a nonexistent username is provided for fetching GitHub data, the application should display a clear and informative error message, informing the user that the username does not exist or cannot be found. This will provide users with immediate feedback and guidance on how to proceed.
Steps to Reproduce:
Enter a nonexistent username in the GitHub data fetching feature.
Attempt to fetch data for the provided username.
Observe the lack of response or unclear error message indicating the issue.
Proposed Solution:
Implement robust error handling mechanisms to detect when a GitHub username does not exist or cannot be found. Upon encountering such a scenario, display an error message on the user interface indicating that the provided username is invalid or cannot be located. Additionally, consider providing suggestions or guidance on how users can verify the correctness of the username or troubleshoot the issue.
Additional Notes:
Ensure that the error message is user-friendly and provides actionable information to assist users in resolving the issue.
Test the error handling functionality thoroughly to validate its effectiveness across different scenarios of invalid or nonexistent usernames.
example image

How are you gonna check non-existent usernames?

GitHub API Error Handling for Nonexistent Usernames
When attempting to fetch data for a GitHub user, if the provided username does not exist, the GitHub API returns a 404 status code with a message indicating "Not Found." This response means that the username does not match any user in GitHub's database.
To handle this scenario, you should add validation in your application to check for the 404 status code. When this status code is encountered, your application should display a user-friendly error message informing the user that the provided username does not exist.
API Response for Nonexistent Username
{
  "message": "Not Found",
  "documentation_url": "https://docs.github.com/rest/users/users#get-a-user"
}


sqlite-jdbc-*.jar can't support aix system

No native library found for os.name=AIX, os.arch=ppc64. Details are as follows:
Caused by: java.lang.Exception: No native library is found for os.name=AIX and os.arch=ppc64. path=/org/sqlite/native/AIX/ppc64 at org.sqlite.SQLiteJDBCLoader.loadSQLiteNativeLibrary(SQLiteJDBCLoader.java:335) at org.sqlite.SQLiteJDBCLoader.initialize(SQLiteJDBCLoader.java:64) at org.sqlite.core.NativeDB.load(NativeDB.java:63) at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:235)
Questions are as follows:

Is the file path **/native/AIX/ppc64/libsqlitejdbc.so?
Could you please tell me how I compile SQLite to get  libsqlitejdbc.so?

Thanks vevy much~

Please help me. Thanks

This is a dup of #588.


Right Alt key removes newlines

System Details

OS: Manjaro Linux
Thorium Version 117.0.5938.157 (Official Build) (64-bit)

Problem
After pressing AltGr (right Alt) in a text box, the newline characters are removed.
Additional Notes
Same things happened on Chromium.

If chromiun also has this problem, it may be intentional by Google, or there may be a bug.
It is recommended that you report this issue again on crbug.
@GrimPixel


I want to join this community

Thank you for your interest in joining this community, @damtrix but this is the invitation link
Register Here


Personal Site Version 4

Splits blog and main site into 2 projects
Includes both in one repo
New blog uses markdown posts

Just need to change the deployments on Vercel for this to work correctly


Request - option to toggle off active navbar item

Could a switch option be added into the theme settings to disable the active navbar items please?  It only works on a few items for me and I'm not a fan of how it looks on most of the themes.

Can do.

is this caused by my action ?

Haha, yeah. No biggie, we make both happy :P

@Hellowlol Awesome, thanks.
@bakman2 I'm not sure but the only item that actually highlight when active for me is Plex.  The other items I have it doesn't work on are Sys Info, vnStat, Samsung Remote, Sonarr, Newznab, Torrents or CouchPotato.

I havnt tested this myself, i checked for syntax errors. I think it would be very nice if it used the same color as when you "hover" over them.

I see it :( didn't fail for me as the menu items matched the id's. But I see that my "stats" has the ID of vnstat. Have to figure out a different way.
@Hellowlol I see an array of the menu items being generated is that coming from python somewhere ?

Its coming from the db as the menu can me reordered.

Is it possible to determine the requested page ?
then it should be possible to build the array with an additional class in this part ?
 %if s.get(m.get('id')+'_enable'):
            menus['nav-${m.get('id')}'] = '
${m.get('name') if s.get(m.get('id')+'_name').strip() == "" else s.get(m.get('id')+'_name')|h, entity}
';

        %endif

I dont think so, i think we have to solve this with js.

i got an idea on the navbar thingy

@Hellowlol in default.js i see
    path = window.location.pathname.split('/')
    $('#nav-'+path[1]).addClass('active')

this doesn't appear to work, not sure why, maybe move to the bottom ?

Hmm, i cant remember what that was used for, i tought it was for the tabs or something :P You can try D

haha it's funny you posted that code as that was my plan, i was gonna get the last part of the url and then use that as the class just like this code does. that said i don't think this would work for me, as i have remote.cyanlabs.net/htpc/NAME so it wouldn't be the second /

path = location.href.substring(document.location.href.lastIndexOf( '/' ));
$('#nav-'+path).addClass('active')
would work on all cases.

or
path = window.location.href.split("/").pop();
$('#nav-'+path).addClass('active')

strip of the last /
 path = window.location.href.split("/").slice(0, - 1).pop()
   $('#nav-'+path).addClass('active')

yep noticed that :D

making PR to fix nav active

@raysguy Is this still needed? I havnt seen any themes that are affect by this (only checked some.)

The ones it doesn't look good on are moonlight, plexibility and sublime as the text color when highlighted doesn't match the text color when hovering.

Hmm, any reason why we cant change that?

Lol I have no idea :P I'll have a look if the feature is updated to see how it looks on those.

Well make it look like hover? I think that looks nice.

Hmm, i cant remember how that looks on tabs / pills

@raysguy I have finally found a solution that im happy with regarding the cache. I have never used so much time on such a small change. It was to slow, fast enough and ate all my memory. I did a small test on 18500 images and it took 160ish sec and only used 90MB. Ill start on this request after its merged.

Awesome that sounds great! :)

On 12 Aug 2015, at 12:07, Hellowlol<EMAIL_ADDRESS>wrote:
@raysguy https://github.com/raysguy I have finally found a solution that im happy with regarding the cache. I have never used so much time on such a small change. It was to slow, fast enough and ate all my memory. I did a small test on 18500 images and it took 160ish sec and only used 90MB. Ill start on this request after its merged.
—
Reply to this email directly or view it on GitHub https://github.com/Hellowlol/HTPC-Manager/issues/302#issuecomment-130262592.

https://github.com/Hellowlol/HTPC-Manager/blob/master2/interfaces/default/js/default.js#L296

There is now a option in the general settings page where you can turn it off.


Branch ppp

Update kaiwei's Project Portfolio Page

Codecov Report

Merging #83 (ffd6458) into master (4f7660a) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #83   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f7660a...ffd6458. Read the comment docs.