Add support for BigInt in compiled mode

This (MIT-licensed) library seems useful: https://github.com/faheel/BigInt.
One challenge is that we haven't solved #378; where should bigints be allocated? From the stack? From the heap? Sometimes one and sometimes the other? To begin with, let's just allocate them on the heap and let the kernel/OS handle the cleanup for us. For static number constants, this will be the right choice anyway. For dynamically generated bigints, we obviously need something better but we'll keep this as part of the scope of #378 for now.

This (MIT-licensed) library seems useful: https://github.com/faheel/BigInt.

This is also another option:

https://github.com/libtom/libtommath (found referenced at https://github.com/fffaraz/awesome-cpp, as one of the backends for Boost.MultiPrecision)

GMP and and MPFR are also mentioned there, but I'll rule them out for now because of the (LGPL) licensing. It's not completely unlikely that I'll change the license for Perlang (to a split license where the compiler is GPL or AGPL and the stdlib is under another license), but I don't want to include LGPL or GPL:ed code at this stage. I want to retain the possibility to make the stdlib available under the MIT license, since I think this might be the most suitable option.

This is now in place, and works, for a certain definition of "works", but... now that I've fixed #420, I realize how dog slow this stuff is. (or rather, seems to be. Very important note: I haven't benchmarked anything, so I don't know that the https://github.com/faheel/BigInt library is causing this.. but it seems likely, given the discussion in https://github.com/faheel/BigInt/issues/69#issuecomment-1817470516).
Running docs/examples/quickstart/pi.per in compiled mode
$ time ./pi
31415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019278766111959092164201989

real	0m7.413s
user	0m7.413s
sys	0m0.000s

Running docs/examples/quickstart/pi.per in interpreted mode
$ time ~/.perlang/release/bin/perlang pi.per
31415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019278766111959092164201989

real	0m0.165s
user	0m0.081s
sys	0m0.026s

Conclusion/takeaway
Of course, more runs would be needed for any serious benchmarking but we see now already that this is way, way slower than you would expect. Ideally, the compiled version should be faster than my silly C# interpreter. :slightly_smiling_face: But the "problem" so to speak is that the Perlang code (and my interpreter) is likely not the bottleneck at all here; rather the BigInteger implementation in C# (which seems to be much faster than the C++ BigInt library we use).
So what I think we should do:

Use the lesson from the BigInt libary we now have to provide roughly the same C++ API (essentially, bigint.hpp currently in place)
Instead of using the bigint.cpp implementation, back the new BigInt class with data structures suitable for working with the https://github.com/libtom/libtommath. This will take some effort; learning how this C-based library works, what files in it that relevant for us (hopefully not all of it), and so forth.
Evaluate the effort and ensure that we go well below 0.165s for the above pi calulations, in compiled mode. I think a reasonable goal should be something like 0.082s, i.e. 50% faster. Let's see if this achievable or not; at the very least, we should at least not be slower than interpreted mode.

Reopening the issue.

There we go. With the changes in https://github.com/perlang-org/perlang/pull/425 in place, we are now down to something reasonable in compiled mode too. Bear in mind that this is just for calculating 1000 decimals now; the 10 000 decimal version is still not as fast as neither the C# nor JavaScript versions mentioned in our own docs, but I think it's "good enough" for now. If/when we need better performance than this, we'll have to investigate it further.
❯ time docs/examples/quickstart/pi
31415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679821480865132823066470938446095505822317253594081284811174502841027019385211055596446229489549303819644288109756659334461284756482337867831652712019091456485669234603486104543266482133936072602491412737245870066063155881748815209209628292540917153643678925903600113305305488204665213841469519415116094330572703657595919530921861173819326117931051185480744623799627495673518857527248912279381830119491298336733624406566430860213949463952247371907021798609437027705392171762931767523846748184676694051320005681271452635608277857713427577896091736371787214684409012249534301465495853710507922796892589235420199561121290219608640344181598136297747713099605187072113499999983729780499510597317328160963185950244594553469083026425223082533446850352619311881710100031378387528865875332083814206171776691473035982534904287554687311595628638823537875937519577818577805321712268066130019278766111959092164201989

real	0m0.035s
user	0m0.027s
sys	0m0.009s

Fixed in #425.


Unable to run private cluster

Steps:

I start requester node on ubuntu:
bacalhau serve --node-type requester

It prints on console:
To connect another node to this one, run the following command in your shell:
bacalhau serve --node-type=compute --network=libp2p --peer=/ip4/<IP_ADDRESS>/tcp/1235/p2p/QmeKmFPPkBdSxGySxYDf7QZXChXucrmN6LzzXYDZMsPMg3 --private-internal-ipfs --ipfs-swarm-addrs=/ip4/<IP_ADDRESS>/tcp/41799/p2p/QmRf4vs8qjVVdjk8hKQHM8kgUG6s3tafB7ueYKUFiRh8if

I try to connect to requester node in another console in the same host in a different directory.
It always gives error -

ERR github.com/libp2p/go-libp2p@v0.27.8/p2p/host/routed/routed.go:153 > got wrong peer error="routing failure: provided addrs for different peer" [gotPeer:] [logger-name:routedhost] [wantedPeer:QmeKmFPPkBdSxGySxYDf7QZXChXucrmN6LzzXYDZMsPMg3]
07:53:35.165 | WRN pkg/libp2p/host.go:161 > Error connecting to peer, continuing... error="dial to self attempted" [NodeID:QmeKmFPP] [addresses:["/ip4/<IP_ADDRESS>/tcp/1235"]] [peer:QmeKmFPPkBdSxGySxYDf7QZXChXucrmN6LzzXYDZMsPMg3]
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x237b91a]

libp2p doesn't like dialling to the same machine, hence the "dial to self attempted" error.
Options are:

Use a hybrid node bacalhau serve --node-type requester,compute
Use NATS bacalhau serve --node-type requester --network nats
Ensure compute is on a different machine/VM.


Create an abstraction over WebSocket to be used with both dart:io and dart:html

Fixes #190

Thanks for this @xvrh.
Are you aware of the web_socket_channel package, though? You seem to have partially duplicated it, and it might be worth using as it's maintained by the Dart team.

@hacker1024 yeah, I've used this package in some other projects.
I evaluated it for this case but ultimately decided that it's not worth it here.
The abstraction layer and the web implementation is so small that (I think) it's a better strategy than adding a new dependency...
I can change my mind, if you see some other advantages...

Error: Unsupported operation: Platform._environment dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 266:49         throw_ dart-sdk/lib/_internal/js_dev_runtime/patch/io_patch.dart 284:5                      _environment dart-sdk/lib/io/platform_impl.dart 83:17                                             get environment dart-sdk/lib/io/platform.dart 168:59                                                 get environment packages/puppeteer/src/puppeteer.dart 269:71                                         defaultArgs packages/puppeteer/src/puppeteer.dart 112:28                                         launch dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 84:54                   runBody dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 123:5                   _async packages/puppeteer/src/puppeteer.dart 93:25                                          launch packages/MyProject/MyFile.dart 18:35                                               MyFunctionName dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 84:54                   runBody dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 123:5                   _async packages/MyProject/MyFile.dart 17:15                                               MyFunctionName
i get this when try to use it with flutter web , on mac
puppeteer: 2.17.0
fultter: 3.3.6

@Joseph-Nathan it is this test file: https://github.com/xvrh/puppeteer-dart/blob/master/test/connect_on_web_test.dart
Basically the steps are:

Start a browser with puppeteer.launch
Start an http server to serve index.html and script.js
Compile a dart script with dart compile js
Go to index.html with puppeteer (which will load script.js)
index.html page will contain <body x-puppeteer-endpoint="<url of puppeteer">
Let the script in the browser connect back to the host and basically communicate with itself. 🤪

@xvrh  hi ,
i start the browser and get browser.wsEndpoint ,
and used it from my web page by puppeteer.connect(browserWsEndpoint: "xxx" )
now i can open new page , but when i tried to controll the page i got this :
Error: TimeoutException: Navigation Timeout Exceeded: 30000ms exceeded dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 299:10  createErrorWithStack dart-sdk/lib/_internal/js_dev_runtime/patch/core_patch.dart 341:28            _throw dart-sdk/lib/core/errors.dart 116:5                                           throwWithStackTrace dart-sdk/lib/async/zone.dart 1378:11                                          callback dart-sdk/lib/async/schedule_microtask.dart 40:11                              _microtaskLoop dart-sdk/lib/async/schedule_microtask.dart 49:5                               _startMicrotaskLoop dart-sdk/lib/_internal/js_dev_runtime/patch/async_patch.dart 166:15           <fn>


npm install react-sparklines --save - issue

On installing npm install react-sparklines --save
I got error Super expression must either be null or a function, not undefined but when I installed npm install<EMAIL_ADDRESS>--save
the issue got resolved.
Note: Other versions installed:
"react": "^0.14.3",
"react-dom": "^0.14.3",
"react-redux": "4.3.0",
"react-router": "^2.0.1",

I get the same error on Console in Firefox after installing it and importing it.

Same issue for me as well unless using 1.4.2.

Followed https://github.com/borisyankov/react-sparklines/issues/89#issuecomment-322317332
and installed v 1.6.0-- this fixed me.
npm i --save<EMAIL_ADDRESS>


Don't hardcode key mappings

Currently, the key mappings are hardcoded.  This overrides other mappings after calling oatmeal for the first time.

Thanks for the fix.
Could I convince you to drop the key mappings altogether and show how the user can do this themselves?
My problem with the current mechanism is that it's duplicating what I do with lazy:
{
   "dustinblackman/oatmeal.nvim",
   keys = {
      { "<leader>co", mode = { "n", "v" }, desc = "Start Oatmeal session" },
   },
   opts = {
      hotkey = "<leader>co",
      backend = "ollama",
      model = "codellama:latest",
   },
}

so the plugin can be loaded lazily, and I can have a description.
I was looking around to see how other plugins deal with this, and in my brief search, they don't appear to set default mappings.

Got a few example plugins you were looking in to? I'm down to keep things consistent with other plugins.

Sure, I had a look at:

https://github.com/nvim-telescope/telescope.nvim
https://github.com/lewis6991/gitsigns.nvim
https://github.com/folke/trouble.nvim


Size mismatch error

Traceback (most recent call last):
File "msunas.py", line 355, in
main(cfgs)
File "msunas.py", line 310, in main
engine.search()
File "msunas.py", line 83, in search
candidates, c_top1_err_pred = self._next(archive, acc_predictor, self.n_iter)
File "msunas.py", line 197, in _next
{'n_classes': self.n_classes, 'model_path': self.supernet_path})
File "msunas.py", line 266, in init
n_classes=supernet['n_classes'], model_path=supernet['model_path'])
File "C:\Users\hp\nsganetv2-master\evaluator.py", line 102, in init
self.engine.load_weights_from_net(init)
File "E:\Installed_Softwares\lib\site-packages\ofa\elastic_nn\networks\ofa_mbv3.py", line 211, in load_weights_from_net
self.load_state_dict(model_dict)
File "E:\Installed_Softwares\lib\site-packages\torch\nn\modules\module.py", line 1052, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for OFAMobileNetV3:
size mismatch for classifier.linear.weight: copying a param with shape torch.Size([1000, 1280]) from checkpoint, the shape in current model is torch.Size([10, 1280]).
size mismatch for classifier.linear.bias: copying a param with shape torch.Size([1000]) from checkpoint, the shape in current model is torch.Size([10]).

I had the same issue, I suppose you tried the bi-objective search code with the argument --n_classes=10.
A quick ugly fix which worked for me was changing some lines of code from evaluator.py, after initialization of init variable:
init = torch.load(model_path, map_location='cpu')['state_dict']
Init is a dict having a classifier with 1000 outputs:
classifier.linear.weight torch.Size([1000, 1280])
classifier.linear.bias torch.Size([1000])
Here is the place you may do some changes. Just for illustration purposes, modifying
init['classifier.linear.weight'] = init['classifier.linear.weight'][:10]
init['classifier.linear.bias'] = init['classifier.linear.bias'][:10]

resolves your problem.

I had the same issue, I suppose you tried the bi-objective search code with the argument --n_classes=10.
A quick ugly fix which worked for me was changing some lines of code from evaluator.py, after initialization of init variable:
init = torch.load(model_path, map_location='cpu')['state_dict']
Init is a dict having a classifier with 1000 outputs:
classifier.linear.weight torch.Size([1000, 1280])
classifier.linear.bias torch.Size([1000])
Here is the place you may do some changes. Just for illustration purposes, modifying
init['classifier.linear.weight'] = init['classifier.linear.weight'][:10]
init['classifier.linear.bias'] = init['classifier.linear.bias'][:10]

resolves your problem.

Did you encounter this kind of error again when searching? An error is reported when the configured ofa version is ofa 0.1.0-202012082159. Then try ofa 0.0.4-2012082155 but the same error still occurs.
Traceback (most recent call last):
File "msunas.py", line 8, in
from evaluator import OFAEvaluator, get_net_info
File "/data8T/nsganetv2-master/evaluator.py", line 8, in
from codebase.networks import NSGANetV2
File "/data8T/nsganetv2-master/codebase/networks/init.py", line 1, in
from ofa.imagenet_codebase.networks.proxyless_nets import ProxylessNASNets, proxyless_base, MobileNetV2
ModuleNotFoundError: No module named 'ofa.imagenet_codebase'

@euminds - I had plenty of imports errors related with ofa.
I solved them by looking at their github repo: https://github.com/mit-han-lab/once-for-all
If you have the last version of ofa, it's easy to look to your required files.
In your specific case,
from ofa.imagenet_codebase.networks.proxyless_nets import ProxylessNASNets, proxyless_base, MobileNetV2
shoud be replaced with
from ofa.imagenet_classification.networks.proxyless_nets import ProxylessNASNets, proxyless_base, MobileNetV2
If you check the github repo, you see that there is no folder "imagenet_codebase", but there is a folder "imagenet_classification", which contains what you need

@euminds - I had plenty of imports errors related with ofa.
I solved them by looking at their github repo: https://github.com/mit-han-lab/once-for-all
If you have the last version of ofa, it's easy to look to your required files.
In your specific case,
from ofa.imagenet_codebase.networks.proxyless_nets import ProxylessNASNets, proxyless_base, MobileNetV2
shoud be replaced with
from ofa.imagenet_classification.networks.proxyless_nets import ProxylessNASNets, proxyless_base, MobileNetV2
If you check the github repo, you see that there is no folder "imagenet_codebase", but there is a folder "imagenet_classification", which contains what you need

Thanks，I will try

@euminds - I had plenty of imports errors related with ofa.
I solved them by looking at their github repo: https://github.com/mit-han-lab/once-for-all
If you have the last version of ofa, it's easy to look to your required files.
In your specific case,
from ofa.imagenet_codebase.networks.proxyless_nets import ProxylessNASNets, proxyless_base, MobileNetV2
shoud be replaced with
from ofa.imagenet_classification.networks.proxyless_nets import ProxylessNASNets, proxyless_base, MobileNetV2
If you check the github repo, you see that there is no folder "imagenet_codebase", but there is a folder "imagenet_classification", which contains what you need

Thanks. I will try.


Replace log with slog

What 🕵♀
This PR replaces log with slog, a structured logger that will soon become part of the standard Go library.
Why 🔑
Makes logging more performant and tunable.
Testing 🧪

[x] go test ./... from root
[x] go run ./examples/dnsregistry/main.go from root

# Demo

@richardartoul slog has an option for adding the source code position on every log. I've never used that because I don't find it necessary. But would you like to activate it or set a flag for activating it?

Looks good! Just have some nits.
Thanks for doing this!


Project Save / Load functionailty

Is there a proper way on how to get the state of the application as JSON object.

Looked through the source code and I'm afraid there is none. It quite heavily uses angular, scopes, drag/drop via jquery-ui etc that I'm in doubts it's possible.

Agree, any kind of simple retention of settings would be amazing.

Hi all,
saving/loading settings is one of the goal of our fundraising campaign, have a look here:
https://igg.me/at/rawgraphs2
If we manage to reach the goal, it will be one of the functions (among many others) that will be added.

now supported in v2


Visiting a "Favorite" from the home page hangs

Steps to recreate

Make sure you have a DAO created and favorited
Visit the home page.
Click one of your favorites, see that the DAO never loads.
This is also true with the "back" button. If you go to a DAO details, visit the home page and click "back" button, it doesn't load.

likely and issue with DAOController state


some questions about Paper "Guided Collaborative Training"

great work. Thanks for your amazing codebase.
I have some questions about this paper "Guided Collaborative Training for Pixel-wise
Semi-Supervised Learning"
1.I'm wondering if I can just use max score of a pixel as an evaluation criterion without Flaw Detector in semantic segmentation task?  If so, how would it work if I use score directly, have you ever done such experiment?

Is Flaw Correction Constraint forcing the error to 0 to correct the result of semantic segmentation?  This loss, not quite understand what it means.

Hi, thanks for your attention.
For your questions:
Q1: I'm wondering if I can just use max score of a pixel as an evaluation criterion without Flaw Detector in semantic segmentation task? If so, how would it work if I use score directly, have you ever done such experiment?
Sorry that we were not try it since we we also considered the regression tasks ("max score" is unavailable in the regression tasks).
Q2: Is Flaw Correction Constraint forcing the error to 0 to correct the result of semantic segmentation? This loss, not quite understand what it means.
Yes,  Flaw Correction Constraint can correct the result of semantic segmentation.
For this point, you can check Sec 3.4 in our paper. We tried to compare Flaw Correction Constraint with the GAN loss to analyze its ability. Please feel free to ask more detailed questions about this.

Yes, I agree with you. Using the classification score in semantic segmentation is more straightforward.
In GCT, we do not use such a solution because we want to generalize it to the regression tasks.

Thanks for your patient reply! Best

Hi, sorry to bother you.  I have a question about codebase in GCT.
In this paper, two networks have different initializations to form the perturbations between them. But I can't find where are the different initialization settings.
        self.l_model = func.create_model(model_funcs[0], 'l_model', args=self.args)         self.r_model = func.create_model(model_funcs[1], 'r_model', args=self.args)
`    model = mclass(**kwargs)
model = torch.nn.DataParallel(model)
model = model.cuda()
logger.log_info('  ' + '=' * 76 + '\n  {0} parameters \n{1}'.format(mname, model_str(model)))
return model`

Could you tell me where is it?

You can find the random initailization step in the model defination. For example: https://github.com/ZHKKKe/PixelSSL/blob/ce192034355ae6a77e47d2983d9c9242df60802a/task/sseg/module/deeplab_v2.py#L78-L79
Initailization usually done by the nn.Module class, not in the SSL framework.
Thanks.

So both models are randomly initialized to ensure to be different?

Yes, exactly.


Rearrange slide panel to change tab order

Summary
Addresses issue #1272 from terra-core. Divs are rearranged so that tabbing will focus main panel first, slide panel second.
Closes cerner/terra-core#1272

@tbiethman Do you know if there is any concern with switching DOM order of panel and main content?

The DOM position of the panel should be dependent upon the panelPosition value. I would assume that if the panel is in the start position, the existing DOM order would be preferred.

Functional verification completed. The slide panel examples now have the correct tab order.