WARC writer incorrectly adds extra line in response records between HTTP headers and payload content

(reported by @wumpus, thanks!)
The WARC writer adds a redundant line between the HTTP headers and the payload content of WARC response records. This contradicts the HTTP standard which requires exactly one single empty line between the response header fields and the payload content.
This bug affects all WARC files (resp. the response records) in the Common Crawl August 2018 crawl archives. This may cause the following problems when the WARC files are processed:

a WARC reader/parser assumes only a single empty line, the extracted payload content starts with "\r\n". Leading new lines are usually ignored by HTML parsers, but will fail other parsers for PDF, office documents, etc.
the payload length indicated in the HTTP Content-Length header (optional) is off by 2: the payload includes the leading "\r\n" while the header value does not include it.

The HTTP headers are "recorded" by protocol plugins. A check of the plugins shows that they do not agree regarding

the number of trailing line breaks (2 => one empty line)
the type of the line break (\r\n or n)
even the same plugin may record the headers differently for request and response headers

Response
Request
Plugin
Nutch version

2 \r\n
2 \r\n
protocol-okhttp
1.15

1 \n
2 \r\n
protocol-http
1.15

(not
supported)
protocol-httpclient
1.15

1 \r\n
2 \r\n
protocol-http
1.6 CC

This bug is caused by an upgrade to Nutch 1.15 and a change from protocol-http (Common Crawl's fork) to protocol-okhttp. The latter records both request and response headers with a trailing empty line. The WARC writer should not add an extra empty line then. Ideally, the protocol-http should be fixed to be consistent regarding line breaks.

While resolving a separate issue I noticed that the WARC-Payload-Digest and WARC-Block-Digest values for response records in the CC-MAIN-2018-34 collection appear to be broken. They should be identical for 'vanilla' response records, but the WARC I looked only the WARC-Block-Digest matched the actual payload bytes.
I then came across this issue, and am wondering whether this is down to the differences in whitespace handling? Or perhaps I should open this as a separate issue?

Hi @anjackson, thanks for remembering me about the digests. The WARC files of the September crawl are correct now. If time I'll want to unify the header format for all Nutch protocol plugins. Eventually, we'll fix the WARC files of the August crawl but that's cumbersome as the URL index will be wrong for a couple of days until all WARC files are rewritten and reindexed.

Format unification of stored HTTP headers tracked in NUTCH-2657.

NUTCH-2657 is fixed
issue document in August 2018 crawl announcement


Nightly Tests have stopped or are lagging

Checking the nightly tests is looks as though the last test ran on May 5th. When I run locally there are several failing tests and getting this information the day of makes troubleshooting problematic commits from TF core a lot easier.
https://source.cloud.google.com/results/invocations/7f864214-a363-4ccd-9887-28f752d0e4a4/targets/tensorflow_addons%2Fubuntu%2Fcpu%2Fpy2%2Fcontinuous/log
Could you take a look when you're free @av8ramit @karmel

FYI it looks like today's fail is from missing SelectV2 gradient and it got rolledback so build should pass tomorrow.
https://github.com/tensorflow/tensorflow/pull/15982

@qlzh727 -- is this related to the changes you made to the build frequency?

Humm, the change I made for changing the project type is submitted on Apr 30, and I still can see new build is triggered after May 5, eg https://source.cloud.google.com/results/invocations/ee86873d-b266-40e6-8ebd-64e8a8273cd1/targets.

Btw, the tests are triggered now by change commit to the master branch on tfa, so that any change that is breaking tfa can be quickly picked up. That said, if there isn't any change, then it will not run.

Hmm, yes, it looks like tests are running again? Was it just the SelectV2 that was the problem, @seanpmorgan ?

Yeah there is no issue with the build at the moment. So if I understand correctly tests will only be ran daily if there are changes to our master branch? I'm not sure that's sufficient as a large portion of our code breaks have been because of the daily changes to TF core master branch.

I changed it to only run after any change in the tfa repo, and skip the nightly run. Let me update it to run with in both CI fashion and with nightly trigger.

Should be fixed now.

Thanks!


Change \A and \z to ^ and $ in regex (with tests)

Adds test cases to https://github.com/joecorcoran/judge/pull/36.
thanks @jokklan!

Also, fixes https://github.com/joecorcoran/judge/issues/32


Add missing backslashes in prometheus_ctl

After the upgrade to v26.0.0 I noticed that web.external-url flag shows empty string in Prometheus Command-Line Flags page. In my environment all command line flags after the storage.tsdb.no-lockfile flag are not initialized during the start.
This also affects Alertmanager integration with Prometheus.

@frodenas Could you please review this PR?
I would also mark v26.0.0 as a pre-release.


Rohit2b/helm commands

Document helper Helm commands

Thanks @arodoni ! Incorporated your review suggestions. Could you re-review?


Duplicate entries in the Window menu on macOS

Issue Type: Bug
I am using native tabs and noticed that there are two "New Tab" entries, as well as all the other tab/window options being duplicated.
This is not the case when the following line in my settings.json is commented out.
  // "window.nativeTabs": true,

Screenshot of issue when nativeTabs is true:

VS Code version: Code 1.28.2 (7f3ce96ff4729c91352ae6def877e59c561f4850, 2018-10-17T00:18:43.347Z)
OS version: Darwin x64 18.0.0

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz (8 x 2200)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: enabledrasterization: enabledvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
2, 2, 2

Memory (System)
16.00GB (1.85GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (11)

Extension
Author (truncated)
Version

chef
Pen
0.7.0

unique-lines
bib
1.0.0

pgformatter
bra
1.5.1

shell-format
fox
1.1.3

gc-excelviewer
Gra
2.1.26

python
ms-
2018.9.1

vscode-docker
Pet
0.3.1

simple-ruby-erb
vor
0.2.1

HCL
who
0.0.5

change-case
wma
1.0.0

markdown-all-in-one
yzh
1.6.2

It was addressed on https://github.com/Microsoft/vscode/pull/59793.
I think it will be available in November's release

Cool, I'll verify when that comes out. Thanks!

This issue duplicates #57421


Inconsistent npm name compleition

I'm not sure if this is npm or us, but verifying https://github.com/microsoft/vscode/issues/120315 I noticed some oddness.

Type @types/react-s in a package.json dependencies key and trigger completion
The only suggestion is @types/react-s-alert, rather than @types/react-select

This seems arbitrary, @types/react-select is vastly more popular than @types/react-s-alert.

It's what's coming from npm. I give them a query string and they come back with matching stuff.
I could try to massage the queries, but I'm hesitating to do that.
No plans for now


Remove ADAI Tutorials

PR to delete the tutorials after moving to own repository

I need to enlist Joshua Margo's help to review this.  As we are deleting tutorials with this, I want to ensure I understand any extra verification checks that we need to perform. I'll try to do that today. Stand by, please.


Resolve terminal warnings

The app current produces some non-breaking warnings in the terminal. Resolve them!

Fixed in #27.


tap_telemetry default

There's gonna be a new property tap_telemetry. installed_for_vmware_internal_use. Perhaps we can default this to true?

Added a commit https://github.com/atmandhol/tappr/commit/f6ca7b1c6298e3233349943a706e50c05d83e30c to fix it.


babel-eslint is now deprecated

babel-eslint is now @babel/eslint-parser and has moved into the Babel monorepo.
https://github.com/babel/babel/tree/main/eslint/babel-eslint-parser

Duplicate #101


Allow services (udp, graphite, collectd and opentsdb) to select database in each metric

The main goal of this PR is to allow save metrics from UDP protocol in different databases.
I have modified batcher so it scans tags looking for a "database" tag. If found, it uses it a the destionation database for this particular point.
Batcher stores points in arrays and then deliver in batchs for writting. To avoid modifying Point struct, now it stores each point inside a map of arrays, with the keys being the different databases.
Services loop over this map saving points in their databases.
This is not a final PR, is just to discuss if this approach is acceptable.
Some things to review:

now batcher arrays inside the map doesn't have max size
when batcher is asked to flush it checks if there is something to flush measuring the length of the array. Now len() return the number of keys. Change it to scan the arrays of all keys?
performance
fix tests

This PR will solve #5156
Also probably this one will be easy #3329

Thanks @adrianlzt.  Unfortunately, defining a special database tag won't work because it now makes database unusable for anyone that might be using it already.  To support this functionality, we need something a little more general.
I see the utility of being able to have a single endpoint with points routed to different databases, but I think it would be better to open a feature request issue and start a discussion of how this could be implemented.
Some ideas that might address this now would be:

Define continuous queries that select points into a new database that match your database tag.  e.g. (create continuous query foo_route on test begin select * into "foo"."default".cpu from cpu where "database"='foo' end
Directly run select info queries

Another option might be to develop a matching framework that can match on measurement and tags and return a database and/or retention policy to use for the point.   Something like this could be integrated into each endpoint and work off of the parsed points perhaps.

I guess that problem could be easily solved using a different name for the tag, like @db or _DB_.
I done this trying to be as simple as possible to not impact in UDP performance.
The matching framework sounds good, but maybe killing performance?
Thanks


Error downloading gfpgan model

Describe your environment

GPU: cpu
CPU arch: x86_64
OS: macOs
Python: [Anaconda/miniconda/miniforge/pyenv/other (explain)]
Branch: development

Describe the bug
running python3 scripts/preload_models.py fails to download gfpgan model and print following error message:
downloading gfpgan model file...
Error loading GFPGAN:
Traceback (most recent call last):
  File "/Users/jakubkolcar/custom/InvokeAI/scripts/preload_models.py", line 95, in <module>
    urllib.request.urlretrieve(model_url,model_dest)
  File "/Users/jakubkolcar/miniconda3/envs/ldm/lib/python3.10/urllib/request.py", line 251, in urlretrieve
    tfp = open(filename, 'wb')
FileNotFoundError: [Errno 2] No such file or directory: 'src/gfpgan/experiments/pretrained_models/GFPGANv1.3.pth'

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

closing, it was by me having corrupted conda environment


Added setter for tab text colors

Fixes #10

Hi, @Numan1617
Thank you for PR
Please pull request of only 1c4228c and 3789af2.
.gitignore file I will fix later.

Hi @ogaclejapan,
I've just forced pushed to update the PR without the gitignore changes.

@Numan1617 Thank you :+1:

No worries. Whats the usual release cycle to maven for changes?

Hi, @Numan1617
For now, planning to release the next version within one day.


CLI progress bar causes failure on Linux

Problem using the CLI on Linux machines. Per XiaoYun the problem is the progress bar feature.
Start Training
start nni training
System.ArgumentOutOfRangeException: Count cannot be less than zero. (Parameter 'count')
at System.String.Ctor(Char c, Int32 count)
at Microsoft.ML.CLI.FixDuriationTimeProgressBar.get_ProgressBar() in //src/mlnet/Utilities/FixDuriationTimeProgressBar.cs:line 44
at Microsoft.ML.CLI.FixDuriationTimeProgressBar.Draw() in //src/mlnet/Utilities/FixDuriationTimeProgressBar.cs:line 72
at Microsoft.ML.CLI.Utilities.PBarConsolePrinter.Print(String msg) in //src/mlnet/Utilities/PBarConsolePrinter.cs:line 43
at Microsoft.ML.CLI.Runners.AutoMLRunner.Instance_DiagnosticDataReceived(Object sender, DataReceivedEventArgs e) in //src/mlnet/Runners/AutoMLRunner.cs:line 109
at Microsoft.ML.ModelBuilder.AutoMLService.AutoMLServiceLogger.Log(AutoMLServiceLogLevel loglevel, String msg) in //src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLServiceLogger.cs:line 45
at Microsoft.ML.ModelBuilder.AutoMLService.AutoMLServiceLogger.Info(String msg) in //src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLServiceLogger.cs:line 27
at Microsoft.ML.ModelBuilder.AutoMLService.Experiments.LocalAutoMLExperiment.ExecuteAsync(IDataView trainData, IDataView validateData, ColumnInformation columnInformation, CancellationToken cancellationToken, CancellationToken timeout) in //src/Microsoft.ML.ModelBuilder.AutoMLService/Experiments/LocalAutoMLExperiment.cs:line 65
at Microsoft.ML.ModelBuilder.AutoMLEngine.StartTrainingAsync(TrainingConfiguration config, PathConfiguration pathConfig, CancellationToken userCancellationToken) in //src/Microsoft.ML.ModelBuilder.AutoMLService/AutoMLEngineService/AutoMLEngine.cs:line 160
at Microsoft.ML.CLI.Runners.AutoMLRunner.ExecuteAsync() in //src/mlnet/Runners/AutoMLRunner.cs:line 88
at Microsoft.ML.CLI.Program.TrainAsync(TrainingConfiguration trainingConfiguration, PathConfiguration pathConfig, AutoMLServiceLogLevel logLevel) in //src/mlnet/Program.cs:line 348
at Microsoft.ML.CLI.Program.AutoMLCommandRunner(AutoMLCommand command, Boolean skipGenerateConsoleApp) in //src/mlnet/Program.cs:line 329
at Microsoft.ML.CLI.Program.<>c.<b__4_1>d.MoveNext() in //src/mlnet/Program.cs:line 116
--- End of stack trace from previous location ---
at System.CommandLine.Invocation.CommandHandler.GetExitCodeAsync(Object value, InvocationContext context)
at System.CommandLine.Invocation.ModelBindingCommandHandler.InvokeAsync(InvocationContext context)
at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass23_0.<b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Microsoft.ML.CLI.Program.<>c__DisplayClass4_0.<b__9>d.MoveNext() in /_/src/mlnet/Program.cs:line 290
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__24_0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass11_0.<b__0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<b__10_0>d.MoveNext()
--- End of stack trace from previous location ---
at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass14_0.<b__0>d.MoveNext()

@LittleLittleCloud FYI I filed this issue for tracking

@LittleLittleCloud Is this resolved?

Nope it's not,, I haven't got time to look into it yet.

@LittleLittleCloud are we okay to close this issue since dotnet/machinelearning-tools#1425 has been merged?

Yep


[Q] How to log at different frequencies and 2 different training runs

I have the following code structure
for ep in range(num_epoch):
    # do some training stuff
    wandb.log({"some_metric": some_metric})
    if ep % 5 == 0:
        wandb.log({"another": another})

for ep in range(num_epoch):
    # do some different training stuff say stage 2 training
    wandb.log({"some_metric_2": some_metric_2})

I want to log the above metrics in such a fashion. But the problem that I face is the following - the logger is maintaining a global step counter so that is causing inconsistencies in logging which can be obviously seen from the above code. I tried instantiating different loggers by run = wandb.init(), but that does not seem to be working.

Hi @meghbhalerao, you can use the step parameter in your log function: wandb.log({'loss': 0.2}, step=step) in order to log at different steps. Here's a docs page with more information on how to do this: https://docs.wandb.ai/guides/track/log#stepwise-and-incremental-logging

Hi, thanks for your help. But, I dont think this solved my issue.
The logs also mention that the step size must be a monotonically increasing natural numbers here - https://docs.wandb.ai/guides/track/log#stepwise-and-incremental-logging - so I can't log at any given step that I want.
Please let me know if I am missing something or if I have misunderstood something.

You're right, we currently don't allow for non-monotonically increasing values for our step counter. Can you tell me what the use case to have this feature would be?

Hi @lesliewandb,
Thanks for your post and sorry for the late response. So there are multiple use cases that I have come across -

For example I have 2 quantities that I want to log, say a and b. I want to log a at every training epoch, and say I want to log b every n iterations. It will be very useful if we can instantiation 2 wandb objects within the same process and log them at whatever times that we want. I think the implementation would be very simple like for example -

run1 = wandb.init(some_params)
run2 = wandb.init(some_other_params)

and then I can log these 2 runs at whatever place I want using run.log(whatever_dict)
I need this use case for my project where I want to log the loss at every training epoch and I want to log some images which I am generating at every 1000 epochs since the image takes up a lot of space and it is not needed to store tens of thousands of things. Now, I am doing some hack like I am logging images at every epoch % 1000 iterations using wandb.log(image_dict) and at the remaining iterations I am logging images as None - which is a very hacky way to do it.

Another use case for my project is that my machine learning training happens in multiple stages - for example, I run multiple training loops within the same python script (and I think this is very common in machine where multi stage training is done.) - in this case what happens is that suppose I have n1 epochs in the first training loop and n2 epochs in the second loop and so on, wandb logs the second training loop from n1-th iteration through n1 + n2 th iteration rather than logging from  0 to n2-th iteration - which makes it very hard for comparison when I have to compare 2 runs.

Another use case which commonly comes up and is also coming up in my project is when I trying to log different learning rates. Some learning rates have to naturally be logged at every iteration (for example cyclic  learning rate schedule in pytorch. - https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.CyclicLR.html), while other LR schedulers need to be logged at every epoch which is in general different than the number of iterations in a general mini batch setting.

Summing up, I think the all the use cases can be solved if support can be added to instantiate different runs using wandb.init()and then logging those runs whenever the user wants. And I think the runs can be identified using some unique identifier which can be passed in as the argument to wand.init(). I think there is already an argument like name or run id which can serve as a unique ID for a given run (https://docs.wandb.ai/ref/python/init)
Thanks and please let me know if I am missing or if anything is unclear. I would also be happy to contribute if there is any help needed in implementing this feature.
-- Megh

Thank you for your elaborate response @meghbhalerao! I'll make a ticket for this and let you know if there are any updates on it


Py-earth and numpy 1.25

I update numpy to the 1.25.1 version and now I got an error with py-earth (np.int is deprecated ...).
Does anyone try to use py-earth with numpy 1.25.1 and/or know how to modify the code to avoid the error ?
Thank you very much.
Have a good day.

Here is the error I get when I try the example plot_v_function.py with numpy 1.25.1 :
Traceback (most recent call last):
  File "/home/pascal/Bureau/plot_v_function.py", line 12, in <module>
    from pyearth import Earth
  File "/usr/lib/python3/dist-packages/pyearth/__init__.py", line 6, in <module>
    from .earth import Earth
  File "/usr/lib/python3/dist-packages/pyearth/earth.py", line 1, in <module>
    from ._forward import ForwardPasser
  File "pyearth/_forward.pyx", line 1, in init pyearth._forward
  File "pyearth/_basis.pyx", line 12, in init pyearth._basis
  File "pyearth/_types.pyx", line 3, in init pyearth._types
  File "/home/pascal/.local/lib/python3.10/site-packages/numpy/__init__.py", line 313, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'int'.
`np.int` was a deprecated alias for the builtin `int`. To avoid this error in existing code, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

Can someone help me ? What can I do ?
Thank you.

Try my new pull request.  It works for me.

Thank you, I'll try it.
Have a good weekend.


git: not found

bundle install fail on gem 'rails', github: "rails/rails"
I think git also need here

Same issue as #11 we don't add any different as official do.
If you need Git by default, install it or create your own repository.


Instructions on how to manually compile and run?

I'm not writing on Rust and not even understanding it's basic syntax, but in case if this repository won't update in time when new Bot API update releases I want to generate schemas manually. But since I'm not a fan of Rust, I actually don't know how to compile and run that code. Could you make a documentation page or something that will be describing steps on how to generate schemas manually?
Just basic stuff like "before compiling make sure that you have at least X.X version of Rust installed, ..." and steps on how to compile the code and run it

if this repository won't update in time when new Bot API update releases

Repository automatically updates every day, so it's always up to date
But if you insist, I will write steps


Ensure the schema and corpus are ready for the diary study

Definition of done

[ ] The environment that the diary study instructs the user to use has an up-to-date schema and corpus

We are still blocked on this. There is one last query to resolve and a bug to fix. @GenevieveHaliburton and @MDunitz are working on this.

We're conducting the diary study on staging instead of prod.


drivers: nsos: set IPV6_V6ONLY to allow IPv4 and IPv6 bind on two FDs

Zephyr allows to bind two sockets separately to IPv4 and IPv6. In order to have that behavior on Linux set IPV6_V6ONLY socket option.
According to 'man 7 ipv6' about IPV6_V6ONLY:
  If  this  flag  is  set to true (nonzero), then the socket is re‐
  stricted to sending and receiving IPv6  packets  only.   In  this
  case,  an  IPv4 and an IPv6 application can bind to a single port
  at the same time.

Fixes: #71243

There is something similar in ncat implementation: https://github.com/nmap/nmap/blob/4ad5e7ca2205ee225bb8ea5a4eb3c182b309b1e1/ncat/util.c#L442-L459

@jukkar Thanks for review. I have created another PR #72341, which should better address the same underlying issue.


Upgrade to latest tower lsp

Changed

Upgrade to tower-lsp 0.9 and tokio 0.2.
Define ServerInfo in InitializeResult to declare the server as tremor-language-server to the LSP client.
Use tokio::sync::Mutex to avoid blocking the async executor.

I tried to follow the instructions in the CONTRIBUTING.md, but there seems to be no contrib/pre-commit script available (probably because of #4). Please do let me know if there's something I should change here to better adhere to the guidelines!
r? @darach @Licenser
Closes #5.

Note that I also did not remove any of the stub LanguageServer trait methods in this pull request, even though they are now optional in the latest version of tower-lsp, but I could do that too if you'd like me to!

Hi Eyal. Thank you for taking the time to upgrade to tower-lsp 0.9. Good point on a pre-commit. We should probably consider one for the language-server. We have one in the core tremor-runtime project but it has now subsumed into the github workflow. That's something we should reconsider and/or fix the guidelines accordingly!

Wow, that was really fast. Thanks for the speedy review, @darach! 🎉 Keep up the good work, everyone.

Thank you so much @ebkalderon ! It is absolutely great that you took the time to make this PR!


Italian language support

Please, make it possible.
Is there any way I can help?
What is the procedure to support a new language?

Italian support has now been added. :)


Rewrite control module/component for TCP/IP

[x] design simple protocol
[x] implement module
[ ] implement component

This currently works on UDP. This protocol is not suitable for that. There are no guarantees that all the packets arrive. Protocol messages should have variable length. The connection should be monitored. Transition to TCP/IP is necessary.

In proof-of-concept UDP implementation it works as:
enum Commands {ENABLE=0, DISABLE=1, DISABLE_ALL=3, ENABLED=4, DISABLED=5, FAILED=6 };

struct control_packet
{
	uint64_t timestamp_us;
	int16_t command;
	int16_t creation_delay_ms;
	char unique_name[MODULE_NAME_WITH0_MAX];
	char call[MODULE_CALL_WITH0_MAX];
};

With:

ENABLE=0, DISABLE=1, DISABLE_ALL=3  component -> module communication
ENABLED=4, DISABLED=5, FAILED=6  module -> component communication

The TCP/IP protocol proposal (v1) for control module/component:
Commands

positive for component -> module
negative for module -> component

enum Commands {ENABLE=1, DISABLE=2, DISABLE_ALL=3, ENABLED=-1, DISABLED=-2, FAILED=-3 };

Message Header
8 bytes with:

timestamp_us, 4 bytes, uint64_t
command, 1 byte,  int8_t
protocol_version, 1 byte, uint8_t, here it's 1
length 2 bytes, uint16_t (payload length, not including header)

Message Payload
Attributes

attribute, 1 byte, uint8_t
length, 1 byte, uint8_t
data, byte size in length, interpretation varies

Attribute Data Types:

string (ascii/utf8, length with '\0', data with '\0' character)
u16 (2 bytes, network order, uint16_t)
i32 (4 bytes, network order, u2 int32_t)

Messages (component -> module)
ENABLE

unique_name (string)
call (string)
creation_delay_ms (u16)

DISABLE

unique_name (string)

DISABLE_ALL
empty payload
Messages (module -> component)
ENABLED

unique_name (string)

DISABLED

unique_name (string)

FAILED

unique_name (string)
return value (i32)

Additional Info

order of attributes in message is mandatory (simple parsing)
downwards compatible (if attribute is not known by application it has to be ignored)

The module side was implented in tcp-control branch, not tested yet

The component side was implented in tcp-control ant tested to work with module side
This only needs some final polishing now.

Merged


A suggestion to avoid CORS and different prod/dev

I really really enjoyed the writeup, but had a suggestion for avoiding the CORS setup and to make prod more similar to dev.  In my flask python I changed the catchall to:
@app.route('/', defaults={'path': ''})
@app.route('/<path:path>')
def catch_all(path):
    # If we're in debug run it all to the javascript dev server
    if app.debug:
        return requests.get('http://localhost:8080/{}'.format(path)).text
    return render_template("index.html")

so that when the flask server is in debug it just proxies straight to the javascript server and you still have all the nice browsersync stuff going on

Indeed, it's a very elegant solution. Thanks @carsongee !
I'll try this and probably include to article.


Separate dev from doc dependencies

This list of dev dependencies is too long:
https://github.com/poliastro/poliastro/blob/1b3da82d82328c5d748959455d93b18b1e740364/pyproject.toml#L70-L98
Found while discussing https://github.com/numba/numba-integration-testing/pull/77

In order to get the docs to build with "make html" I had to install the following dependencies (after following the contributing instructions to build my virtualenv.
sphinx
sphinx-rtd-theme
sphinx-autoapi
nbsphinx
sphinx_gallery
IPython
sphinx_copybutton
sphinx-notfound-page
sphinx-hoverxref
myst_parser
jupytext
ipykernel
czml3
This might help with splitting out the doc dependencies.

Thanks a lot @tjnd89 ! Looks like I opened this issue right on time


Patching MonoBehaviour on all event types fail for types with generic parameter

Fix
Probably this can be fixed if I move this patching to be done either in the preloader, or by using the transpiler, or harmony patch in the patcher side, or combination of both

This was already solved in an old issue


createApp is not a function

D:\Assemble\src\core-vue-app\src\vue-frontend-app\node_modules\vite-ssg\dist\chunks\build.cjs:173
const { routes } = await createApp(false);
^
TypeError: createApp is not a function
at Object.build (D:\Assemble\src\core-vue-app\src\vue-frontend-app\node_modules\vite-ssg\dist\chunks\build.cjs:173:28)
at async Object.handler (D:\Assemble\src\core-vue-app\src\vue-frontend-app\node_modules\vite-ssg\dist\node\cli.cjs:25:3)
PS D:\Assemble\src\core-vue-app\src\vue-frontend-app>

I had the same problem.
Change:
const createApp = ViteSSG(/* Your config */);
export default createApp;

to:
export const createApp = ViteSSG(/* Your config */);

I have this issue but I was already exporting createApp as a named variable in main.ts.

I believe the right solution is to include the following in your vite.config.ts:
  ssr: {
    // TODO: workaround until they support native ESM
    noExternal: ['workbox-window', /vue-i18n/],
  },

We temporarily closed this due to the lack of enough information. We could not identify whether it was a bug or a userland misconfiguration with the given info.
Please provide a minimal reproduction to reopen the issue.
Thanks.

Why reproduction is required

I have a similar problem:
const { routes } = createApp(false);
^
TypeError: createApp is not a function
at build (file:///C:/laragon/www/AITS_website/vuejs1/node_modules/vite-ssg/dist/node.mjs:1001:22)
at async Object.handler (file:///C:/laragon/www/AITS_website/vuejs1/node_modules/vite-ssg/dist/node/cli.mjs:34:5)

I'm having a similar problem:
const { routes } = createApp(false);
^
TypeError: createApp is not a function
at build (file:///C:/laragon/www/AITS_website/vuejs1/node_modules/vite-ssg/dist/node.mjs:1001:22)
at async Object.handler (file:///C:/laragon/www/AITS_website/vuejs1/node_modules/vite-ssg/dist/node/cli.mjs:34:5)


limit to amount of targets

I'm now trying to send 5000 targets to a Bartlett UR10 robot via ROS. But the robot gives no reaction at all. It looks like there is a limit to the amount of targets. Could anyone give some suggestion on such a case?

@Chriszlf: I've moved your issue to ur_modern_driver as I believe you're using that (ur_driver doesn't have a URScript topic).


Re-define numbers in the Functional Combinators section

The previous statement defines 'numbers' as being of type String, Int -- but the Functional Combinators: Map example is using 'numbers' as type List.

+1


Allow multiple constraints in a CHC

Fixes #68 by allowing multiple constraints in CHCs. Note that this is just syntactic sugar - the generated CHCs still have a single constraint, but:

multiple constraints will be merged into a single conjunction
a single constraint will be left unchanged
no constraint will be translated to a constraint of true

Also fixes #70 in the same "syntactic sugar" spirit. We now allow arbitrary SMT terms in the inputs to semantic relations, and we internally transform those to a fresh variable and a constraint. For example,
($a (E.Sem x y (+ x y)))

gets transformed into a CHC with an extra variable q such that:

the body is (E.Sem x y q)
the constraint is (= q (+ x y))