Error when training

Hi,
Thank you for your impressive work.
I've followed all the instructions, including importing fer2013 and ck+ dataset, changing path in ./util/info.json, along with executing preprocess.sh
After that, when I run trainmodel.py, the first epoch seems to run smoothly, but in the end, it throws an error as follows:
897/897 [==============================] - 720s 802ms/step - loss: 3.5670 - accuracy: 0.2134 - val_loss: 2.4353 - val_accuracy: 0.1638
Epoch 00001: val_loss improved from inf to 2.43534, saving model to data/model/Model-01-0.1638.hdf5
Traceback (most recent call last):
File "trainmodel.py", line 140, in
model.fit_generator(
File "/Users//PycharmProjects/engagementRecog/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1975, in fit_generator
return self.fit(
File "/Users//PycharmProjects/engagementRecog/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1230, in fit
callbacks.on_epoch_end(epoch, epoch_logs)
File "/Users/*PycharmProjects/engagementRecog/venv/lib/python3.8/site-packages/keras/callbacks.py", line 413, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "trainmodel.py", line 130, in on_epoch_end
self.training_data = shuffle_dataset(self.training_data)
File "/Users//PycharmProjects/engagementRecog/engagement-detection-master/data/dataset_ops.py", line 35, in inner
return func(args, kwargs)
File "/Users//PycharmProjects/engagementRecog/engagement-detection-master/data/dataset_ops.py", line 82, in shuffle_dataset
data_shape = np.array(args[0]).shape[0]
IndexError: tuple index out of range
Thank you for your help, I appreciate any instruction.

Thanks for raising this issue!
This is a weird one, I'm assuming from the traceback that the error is coming from np.array(args[0]).shape[0], and specifically the args[0] part. This would mean that there is no object being passed to the shuffle_dataset method.
I actually haven't run this code in a while (and the original is on a hard drive somewhere), so for debugging it might be useful to add a few extra statements.
In the DatasetShuffle callback that I've made, could you put an assert statement, like assert self.training_data is not None and assert self.validation_data is not None, as well as a print statement to actually inspect its contents (so right above line 130, after the method header)?
This would be useful for me to see where the error is actually occuring.

Thanks for your reply!
I've added some assert and print as you described:

And here is the output:
Epoch 00001: val_loss improved from inf to 2.21813, saving model to data/model/Model-01-0.2343.hdf5
[]
[]
Traceback (most recent call last):
File "trainmodel.py", line 144, in
model.fit_generator(
File "/Users//PycharmProjects/engagementRecog/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1975, in fit_generator
return self.fit(
File "/Users//PycharmProjects/engagementRecog/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1230, in fit
callbacks.on_epoch_end(epoch, epoch_logs)
File "/Users//PycharmProjects/engagementRecog/venv/lib/python3.8/site-packages/keras/callbacks.py", line 413, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "trainmodel.py", line 134, in on_epoch_end
self.training_data = shuffle_dataset(*self.training_data)
File "/Users//PycharmProjects/engagementRecog/engagement-detection-master/data/dataset_ops.py", line 35, in inner
return func(*args, **kwargs)
File "/Users//PycharmProjects/engagementRecog/engagement-detection-master/data/dataset_ops.py", line 82, in shuffle_dataset
data_shape = np.array(args[0]).shape[0]
IndexError: tuple index out of range
It seems that contents of self.training_data & self.validation_data are empty, but there isn't AssertionError.
I consider if there's anything to do with the dependencies because I'm running the project in an environment different from yours. Do you still remember which version of Tensorflow and Keras (and other packages in requirements.txt) did you use?

Removing shuffle_dataset worked, and I've finished the training.
Thank you so much for your guidance in the issue and assert !
Good luck.😉


Updated README.md to notify users of the node-gyp dependency on Python 2.7

I had some issues installing MARKETProtocol on macOS with Python 3.6.4. The pull request is self-explanatory.

Pull Request Test Coverage Report for Build 221

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.8%) to 92.428%

Totals

Change from base Build 220:
0.8%

Covered Lines:
252

Relevant Lines:
255

💛  - Coveralls


Add sqlite connector and example, and postgres demo db.

Add package scripts and code to build and view dbs, and run examples for postgres and sqlite.

Hi Trey, I did all that as a learning exercise for fastify and drizzle. I
thought the existing example didn't demonstrate much. Getting a properly
documented and working docker-compose config took a while. It was for my
own purposes but I thought others may find it useful. I don't mind if you
don't accept this PR. It served its purpose for me.
Cheers, Paul.
On Wed, 20 Nov 2024, 09:36 Trey Martin, @.***> wrote:

@.**** commented on this pull request.
HI @pvspain https://github.com/pvspain , thank you for the
contributions 🙏 !
We just recently had a contributor open #6
https://github.com/trey-m/fastify-drizzle/pull/6 which we plan to
include in the next release which conflicts with some of what you have here.
Appreciate the level of detail for the instructions around
setup/provisioning, however I have some questions/concerns below.
In postgres/docker-compose.yml
https://github.com/trey-m/fastify-drizzle/pull/7#discussion_r1849135713:

@@ -0,0 +1,31 @@
+version: '3'

I really like the extensive documentation on provisioning the database.
However, do you think consumers of this plugin would find this level of
setup instructions useful? I feel it could fall outside the scope of this
project.
In example/sqlite/index.js
https://github.com/trey-m/fastify-drizzle/pull/7#discussion_r1849140280:

@@ -0,0 +1,47 @@
+const fastify = require('fastify')({ logger: true });

+1 to the separation for different examples.
—
Reply to this email directly, view it on GitHub
https://github.com/trey-m/fastify-drizzle/pull/7#pullrequestreview-2446782771,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAFO4S62J4QTJKKMSG76R4L2BO4N3AVCNFSM6AAAAABR5TD4MKVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDINBWG44DENZXGE
.
You are receiving this because you were mentioned.Message ID:
@.***>


Analyzing uses to much memory

Hi,
I'm not sure if this is normal, but analyzing a corpus of 800MB (ca. 16000 articles) runs out of memory on my machine with 8GB of RAM + 2GB of swap. Can someone with a background in data analysis judge if this is expected?
This might be the main issue for me to scale the database for the physics section of arXiv, as I only have run the analysis on a small portion of it (less than a year for most section, and not all categories that are relevant).
I'll try to profile the memory usage, but I hope the attempt isn't futile. :p

Try decreasing the number of words for the bigrams, right now it's 20,000 but you can probably get away with much less, e.g. 10,000 for a reasonable performance.

That helps a bit, but not enough to stay under 8GB. Gladly I have a bit more available on another machine.


TEST-#3750: Create a variable for minimal partition size.

Signed-off-by: mvashishtha<EMAIL_ADDRESS>
What do these changes do?
Create a variable for minimal partition size.

[x] commit message follows format outlined here
[x] passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] signed commit with git commit -s
[x] Resolves #3750
[x] tests added and passing
[x] module layout described at docs/developer/architecture.rst is up-to-date

@vnlitvinov

Please fix docstring per #3751 (comment), then LGTM

Done.


Deleted files from the untrusted key are also converted !!

Even deleted files from the untrusted key are converted and copied into the clean key.
Is this normal?

Well, I assume they're not deleted, but in the trash, right? If that's right, yes, it is expected, the script search all possible files on the untrusted key.

it's an unexpected behavior, thanks for your answer.

You're welcome.
In practice, Circlean cannot make the difference between a "normal" directory and the "trash" directory, as they are the same thing on the file system, and they differ depending on the operating system you're using.

Thanks again, I still have some questions:

is it possible to prevent CIRCLean from searching files from the Trash (by changing things in the config file for example)?
-if CIRCLean  analyzes 10 files from the Untrusted key, it generates about 10 times more file in the Trusted key, can we reduce the number of resulting file?
The .PDF files without risk are also converted into .pdf_DANGEROUS files, can we change that?

Thank you for your interest int he project.

It is currently not possible to ignore specific directories, the main reason is that CIRCLean has no config file and is static after the SD card is flashed. The goal of the project is to be generic and we expect users with specific usecases to modify the code and flash their own images.

Not really, the resulting files are meta-information extracted from the source files. Again, if you have specific usecases, please describe them here, but it is strongly recommended to adapt PyCIRCLean specifically for them.

The way PDFs are analyzed is by checking if there is active content, such as OpenActions, which can be used to execute malicious content. It turns out to be used a lot in legitimate files, but I'm not aware of reasonable ways to figure tout what an open action does (it wasn't doable automatically last time I checked). But if you know about an other approach, please let me know.

The library used by CIRCLean is here: https://github.com/CIRCL/PyCIRCLean

Thank you for your answer.

Hello again!
To be sure, is there a possibility with CIRCLean(with previous versions..) to DELETE only Suspect content such as OpenActions.. from PDF files or Macros from Office files for example. So the result will be an Office Document without Macros instead of having it as DANGEROUS_FileName. docx_DANGEROUS?

No, there is no reliable way I know of to do that (with office documents nor with pdfs). If you hear of one, please let me know.

Ok, thank you.


prior work

https://github.com/pragya16067/rdflib/commits/RDFStar_Support
and
https://github.com/JervenBolleman/rdflib/commits/rdfstar2
plus associated PR
https://github.com/JervenBolleman/rdflib/pull/3

@gjhiggins as per my comment on your PR #7: thanks for bringing all the links here. We - the student @alooba1 (Song) and I are aware of the prior work but Song's still working on fundamentals of RDF, Turtle, Python Parsing Expression Grammars and so on!
Good to have all these links here now though to ensure we review them when Song's a bit more advanced.


Update aragog wrapper with recent changes

Fixes issue with core_radius keyword
Fixes issue with accessing rheological front as float (not as array)
Updated documentation.

Still need to add aragog as a pyproject dependency - should be handled in another PR.

Nice for the update of the doc !


Runtime WAD file loading

Since WAD files are resources, they can't be loaded from outside the project at runtime.
Need to add a filename check that either loads resource WADs as-is, or invokes the WAD parser to read from a filesystem location and return a transient WAD resource for texture loading.

I'm thinking this should be done in C through a new libwad library for the sake of minimizing the amount of work GDScript has to do - WAD loading is currently a noticeable bottleneck in the texture generation process.

Done, after a fashion.


There is a problem with training a soft teacher

When training, it will suddenly stop, change several GPU services, and the result is still the same

I have the same problem.
Would anyone have some solution or idea?

I have the same problem. Would anyone have some solution or idea?

I have the same problem on a custom dataset. Did you manage to solve it?


Add example to documentation

Closes #124
Adds an example through a Jupyter Notebook.
The notebook is rendered nicely through mkdocs-jupyter.
To inspect the proper rendition of the notebook, please follow these steps:

Clone repository (if not done already).
git clone https://github.com/SINTEF/oteapi-optimade.git
cd oteapi-optimade

Checkout this PR's branch and install all dependencies.
git fetch --all -tp
git checkout cwa/close-124-examples
pip install -U pip setuptools wheel flit
pip install -U -e .[docs]

Execute the local MkDocs server:
mkdocs serve

The local URL should be written to the console at the end of building the documentation and starting the server.
Then through the menu on the left, one can navigate through "Examples" to "Use OTEAPI-OPTIMADE with OTElib".
It should look something like this:

Codecov Report

Merging #125 (10757ed) into main (364ec32) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #125   +/-   ##
=======================================
  Coverage   76.19%   76.19%
=======================================
  Files          15       15
  Lines         605      605
=======================================
  Hits          461      461
  Misses        144      144

Flag
Coverage Δ

linux
76.19% <ø> (ø)

windows
82.50% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Please add mkdocs-jupyter to the requirements. Otherwise you can't start the mkdocs server

In the documentation example, it would be good to include the commands for starting the oteapi services instead of just referring to external documentation. Only one way to start the services needs to be shown here. It is fine to refer to oteapi-services for other options.

Please add mkdocs-jupyter to the requirements. Otherwise you can't start the mkdocs server

I did. Just fixed it. Also, if following the instructions above (installing with the docs extra) it shouldn't be an issue. It was only an issue in the CI/CD workflow, since it uses the dev extra. This is what I fixed.
Are you sure you have checked out the correct branch?

In the documentation example, it would be good to include the commands for starting the oteapi services instead of just referring to external documentation. Only one way to start the services needs to be shown here. It is fine to refer to oteapi-services for other options.

I'd rather not reiterate too much external documentation, since if the external documentation changes, this one will be outdated and unusable for something that does not have anything directly to do with this repository/plugin.

I suppose the middle ground would be to say something like "at the time of writing (15.05.2023) the instructions are..."?

That is fine. I guess the instruction should be along the line of checking out oteapi-services and do
docker-compose pull
docker-compose up -d

Please update the below figure. The filter strategy should come after the data resource strategy in the pipeline:

This is conceptually important. We have the partial pipeline documenting the data source (typically DataResource >> Parse >> Mapping). That should be treated as one unit and always come first in the pipeline. The filter is not a part of the documentation of the data resource, but is a part of our application that selects the subset of all the data in the data resource we are interested in. It should use the initialize() method to specify the query such that only the information we need is fetched from the data resource.

I am trying to follow the example in docs/examples/index.md
pipeline = filter_strategy >> data_resource_strategy
session = pipeline.get()

raises
ApiError: APIError: status=500 Cannot initialize data resource: session_id='session-2f0c89e9-d691-43ed-a1c2-20593345c310' resource_id='dataresource-a3a4bde7-2611-4b1f-a450-8038048a797a'

when calling pipeline.get(). There is unfortunately noe clue about what went wrong. With this not working, it is difficult for me to check the rest of the example. Does it work for you?
A side question: Is the session really returned by the get method? Since it is just a (preferable small) json payload that is anyway transferred between the microservices, it certainly possible to return it. I also think it make sense, but I understood from Thomas that he was against that. Has he changed his mind?
A second side node: We should add an issue in oteapi-services that the full traceback and error message from uncaught exceptions should be included in the response. otelib should be updated to show these error messages. That would make debugging much easier. Maybe such detailed error messages could be seen as a security issue, since they could be used by an attacker to probe the underlying services. If that is considered an issue, the detailed error message could still be made available for privileged users/developers.

By the way, docker logs gave some hints. Seems that I should run docker with the correct directory mounted. It would be great if you could add instructions about how do start docker in the right way for the example to work...
No extra plugin packages provided. Specify 'OTEAPI_PLUGIN_PACKAGES' to specify plugin packages.
[2023-05-15 15:57:40 +0000] [15] [INFO] Running on http://<IP_ADDRESS>:8080 (CTRL + C to quit)
[2023-05-15 16:14:10 +0000] [15] [ERROR] Error in ASGI Framework
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/hypercorn/asyncio/task_group.py", line 23, in _handle
    await app(scope, receive, send, sync_spawn, call_soon)
  File "/usr/local/lib/python3.9/site-packages/hypercorn/app_wrappers.py", line 33, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 276, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 237, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
    return await dependant.call(**values)
  File "/app/app/routers/dataresource.py", line 194, in initialize_dataresource
    session_update = create_strategy("resource", config).initialize(
  File "/usr/local/lib/python3.9/site-packages/oteapi/plugins/factories.py", line 183, in create_strategy
    return StrategyFactory.make_strategy(config, strategy_type)
  File "/usr/local/lib/python3.9/site-packages/oteapi/plugins/factories.py", line 92, in make_strategy
    raise NotImplementedError(
NotImplementedError: The resource strategy 'OPTIMADE' does not exist.

I of cause get the same error message when I try to run the Jupyter notebook in docs/examples/otelib.ipynb.
Please add pip install notebook to the setup instructions.

Repeating the same example twice it quite boring. Can you split it up into two sub-pages, one for each otelib backend.

The example should be run at a part of the CI testing.

Please update the below figure. The filter strategy should come after the data resource strategy in the pipeline:

This is conceptually important. We have the partial pipeline documenting the data source (typically DataResource >> Parse >> Mapping). That should be treated as one unit and always come first in the pipeline. The filter is not a part of the documentation of the data resource, but is a part of our application that selects the subset of all the data in the data resource we are interested in. It should use the initialize() method to specify the query such that only the information we need is fetched from the data resource.

I did this originally, but unfortunately the session object is not returned correctly from the "filter strategy", so I had to switch them around. I believe this issue is being resolved in OTElib, but this is an issue.
Furthermore, retrieving the data from the session object is plain wrong and should not be done like this. But that's out-of-scope of this tutorial to fix.

By the way, docker logs gave some hints. Seems that I should run docker with the correct directory mounted. It would be great if you could add instructions about how do start docker in the right way for the example to work...
No extra plugin packages provided. Specify 'OTEAPI_PLUGIN_PACKAGES' to specify plugin packages.
[2023-05-15 15:57:40 +0000] [15] [INFO] Running on http://<IP_ADDRESS>:8080 (CTRL + C to quit)
[2023-05-15 16:14:10 +0000] [15] [ERROR] Error in ASGI Framework
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/hypercorn/asyncio/task_group.py", line 23, in _handle
    await app(scope, receive, send, sync_spawn, call_soon)
  File "/usr/local/lib/python3.9/site-packages/hypercorn/app_wrappers.py", line 33, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 276, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
    raise e
  File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 237, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 163, in run_endpoint_function
    return await dependant.call(**values)
  File "/app/app/routers/dataresource.py", line 194, in initialize_dataresource
    session_update = create_strategy("resource", config).initialize(
  File "/usr/local/lib/python3.9/site-packages/oteapi/plugins/factories.py", line 183, in create_strategy
    return StrategyFactory.make_strategy(config, strategy_type)
  File "/usr/local/lib/python3.9/site-packages/oteapi/plugins/factories.py", line 92, in make_strategy
    raise NotImplementedError(
NotImplementedError: The resource strategy 'OPTIMADE' does not exist.

Concerning this comment and the previous one.
You need to of course reference this repository in the OTEAPI_PLUGIN_PACKAGES variable to install the plugin. The error message is quite clear: The resource strategy "OPTIMADE" does not exist, meaning it is not installed...

I am trying to follow the example in docs/examples/index.md
pipeline = filter_strategy >> data_resource_strategy
session = pipeline.get()

raises
ApiError: APIError: status=500 Cannot initialize data resource: session_id='session-2f0c89e9-d691-43ed-a1c2-20593345c310' resource_id='dataresource-a3a4bde7-2611-4b1f-a450-8038048a797a'

when calling pipeline.get(). There is unfortunately noe clue about what went wrong. With this not working, it is difficult for me to check the rest of the example. Does it work for you?

You need to install the plugin when running the server...

A side question: Is the session really returned by the get method? Since it is just a (preferable small) json payload that is anyway transferred between the microservices, it certainly possible to return it. I also think it make sense, but I understood from Thomas that he was against that. Has he changed his mind?

This is indeed not the way to work with this, but the only way I know how to at this point in time as there is no endpoint in the service to share files with a client.

A second side node: We should add an issue in oteapi-services that the full traceback and error message from uncaught exceptions should be included in the response. otelib should be updated to show these error messages. That would make debugging much easier. Maybe such detailed error messages could be seen as a security issue, since they could be used by an attacker to probe the underlying services. If that is considered an issue, the detailed error message could still be made available for privileged users/developers.

No. A server should never reveal its internals. Any internal error messages can be picked up there: Internally. As you also experienced from the docker logs.

Repeating the same example twice it quite boring. Can you split it up into two sub-pages, one for each otelib backend.

Yes and no. As this was a quick'n'dirty example, I didn't have capacity to come up with a more exciting example. Furthermore, it shows that there is no (or should be no) difference between the two methods, other than added information.

The example should be run at a part of the CI testing.

This repository already has an end-2-end test that does exactly this. No need to also run the tutorial.
That said, the mkdocs-jupyter plugin allows for executing the notebooks upon building the documentation, hence this could act as a CI test for whether the notebook works or not.
However, since a separate server is required and the notebook references localhost:80 it might be quite tricky to make it work properly. Hence, I'd consider it out-of-scope for this PR and writing this example, but definitely worth further investigation down the road.

I suppose the middle ground would be to say something like "at the time of writing (15.05.2023) the instructions are..."?

That is fine. I guess the instruction should be along the line of checking out oteapi-services and do
docker-compose pull
docker-compose up -d

I'll come up with something. I don't think git clone-ing another repository is the way to go, since the same thing can be achieved by just downloading (or recreating) the docker compose file.

Please update the below figure. The filter strategy should come after the data resource strategy in the pipeline:

This is conceptually important. We have the partial pipeline documenting the data source (typically DataResource >> Parse >> Mapping). That should be treated as one unit and always come first in the pipeline. The filter is not a part of the documentation of the data resource, but is a part of our application that selects the subset of all the data in the data resource we are interested in. It should use the initialize() method to specify the query such that only the information we need is fetched from the data resource.

I did this originally, but unfortunately the session object is not returned correctly from the "filter strategy", so I had to switch them around. I believe this issue is being resolved in OTElib, but this is an issue. Furthermore, retrieving the data from the session object is plain wrong and should not be done like this. But that's out-of-scope of this tutorial to fix.

With respect to this "issue", what should the solution be? Switch it up but then not be able to switch up the code to match or keep it as is with the knowledge that it should be possible to run the pipeline either way around the strategies are placed?

Will merge this now, opening issues where necessary to follow-up. While the issues mentioned by @jesper-friis here are valid, most have been remedied.
The most crucial I see, is the way the pipeline session is (mis)handled as well as how the pipeline setup does not match with the intended way of setting up the specific strategies.


live server

i am install the live server but i can't see the live server option so what is the problem and lot of you tube videos iam watiching but no solution plezz help some details send me on my email<EMAIL_ADDRESS>)send some solution.

Easiest solution is to right click inside your code and you will see the dropdown menu will have Open with Live Server.


Make libnetwork understand pluginv2.

As part of daemon init, network and ipam drivers are passed a
pluginstore object that implements the plugin/getter interface. Use this
interface methods in libnetwork to interact with network plugins. This
interface provides the new and improved pluginv2 functionality and falls
back to pluginv1 (legacy) if necessary.
Signed-off-by: Anusha Ragunathan<EMAIL_ADDRESS>

Companion PR to https://github.com/docker/docker/pull/26860.

@anusha-ragunathan thanks for the vendor updates. Are these changes done to be in sync with docker/docker ?

@mavenugo : vendor update has 2 fold reason. 1. get the docker/docker/plugin/getter package 2. syncup with docker/docker

CI failure on
--- FAIL: TestCreateMultipleNetworks (0.13s)
	bridge_test.go:468: iptables failed: iptables --wait -nvL DOCKER-ISOLATION: iptables: No chain/target/match by that name.

@anusha-ragunathan thats alright. can u pls address the comments ? I will check what is going on with the test failure.

@anusha-ragunathan
Please take care of these failures:
./macvlan_test.go:34: not enough arguments in call to Init
./macvlan_test.go:41: not enough arguments in call to Init
./macvlan_test.go:52: not enough arguments in call to Init

Run make to run all the tests offline.

@aboch : I did, before opening the PR, but ran into errors modprobing ipvsadm although the package is installed and ip_vs is modprobed. A strange error where the test container cannot find modprobe and ipvsadm in $PATH, although its present. I'm debugging that right now.

There were 2 libnetwork test containers that were stuck causing modprobe, ipvsadm PATH errors. Removing those fixed the issue, but now modprobe is unable to open modules.dep.bin. Rebooting host doesnt help. File exists on host.
$ ls -als /lib/modules/3.19.0-6
8-generic/modules.dep.bin
604 -rw-r--r-- 1 root root 618268 Sep 10 10:13 /lib/modules/3.19.0-68-generic/modules.dep.bin

RUN   TestGetFamily
--- FAIL: TestGetFamily (0.00s)
Location:       ipvs_test.go:124
Error:          No error is expected but got no such file or directory
=== RUN   TestService
WARN[0000] Could not load necessary modules for IPSEC rules: Running modprobe xfrm_user failed with message: modprobe: ERROR: ../libkmod/libkmod.c:557 kmod_search_moddep() could not open moddep file '/lib/modules/3.19.0-68-generic/modules.dep.bin', error: exit status 1
WARN[0000] Running modprobe ip_vs failed with message: modprobe: ERROR: ../libkmod/libkmod.c:557 kmod_search_moddep() could not open moddep file '/lib/modules/3.19.0-68-generic/modules.dep.bin', error: exit status 1
ERRO[0000] Could not get ipvs family information from the kernel. It is possible that ipvs is not enabled in your kernel. Native loadbalancing will not work until this is fixed.
--- FAIL: TestService (0.01s)
Location:       ipvs_test.go:91
Error:          No error is expected but got exit status 127
=== RUN   TestDestination
WARN[0000] Could not load necessary modules for IPSEC rules: Running modprobe xfrm_user failed with message: modprobe: ERROR: ../libkmod/libkmod.c:557 kmod_search_moddep() could not open moddep file '/lib/modules/3.19.0-68-generic/modules.dep.bin', error: exit status 1
--- FAIL: TestDestination (0.04s)
Location:       ipvs_test.go:91
Error:          No error is expected but got exit status 127

@anusha-ragunathan
try first to export CIRCLECI then run make

CI failure on https://github.com/docker/libnetwork/issues/1385

LGTM

@anusha-ragunathan thanks for taking care of all the comments. LGTM


Update to 1.18.2 and environment changed to client only.

Glad to see some people actually use this! Sorry I forgot to release this update (even have 1.19 working) partially because I am intending to retire Catherine soon. I have a new mod in the works that uses a config that can be configured from in-game that will replace Catherine, but I just haven't gotten around to finishing it yet. This will probably be the last update for Catherine.


GrBC

@PhotoPaul There is something wrong with you request, because you are expected to add one file, but your request requests to change 2,028 files. I am closing it without merging. Please reopen and comment if you are unsure how to add a request. Please make sure you have read all instructions carefully at the bottom of this page. Thank you!

I must have messed this up. Here is the hopefully corrected pull request #2953.


Dart linux installation instructions should have instructions for pub

https://www.dartlang.org/downloads/linux.html does not mention that pub (and dartfmt etc.) are placed in the location described in https://www.dartlang.org/tools/sdk/. Linking to the latter's instructions would be clarifying.
I came across this following the directions on http://flutter.io/getting-started/ which point to the linux installation instructions.

This issue was moved to dart-lang/www.dartlang.org#1525


Files search: display the attachment thumbnail

This issue concerns the Files tab in the unified search screen, and the Files tab in the room search screen.

Done


First Connect works, the second throws an error

Hi Pires, thanks for providing the obd api it's helped me tons!
I hope this is right way of going about asking for help, if not please redirect me and I will do so. I have had the issue for a while and am out of ideas which is why I turned to you in hopes of figuring out a solution to my problem.
So the first time I connect to OBD I have no issues and read the data just fine. If I however disconnect and try to reconnect, it fails and throws the following error:
W/System.err: com.github.pires.obd.exceptions.UnableToConnectException: Error running 01 46, response: ...UNABLETOCONNECT
After that failed attempt, if I try to reconnect again it works.
After every connection I make sure to close it with the Close Command as such:
CloseCommand closeCommand = new CloseCommand();
closeCommand.run(this.mmSocket.getInputStream(), this.mmSocket.getOutputStream());

Is there something I am not doing correctly? Possibly closing the streams? If so how would I go about doing that?

It may be something with your device.

I've tried it with a number of devices, same error persists. Any ideas what else the issue may be?

01 46 is Ambient Air Temperature. what happened to all the AT commands needed to reconnect?

Perhaps I'm not doing it properly, but upon every reconnect I initialize with the following commands:
new EchoOffCommand().run(this.mmSocket.getInputStream(), this.mmSocket.getOutputStream());
new LineFeedOffCommand().run(this.mmSocket.getInputStream(), this.mmSocket.getOutputStream());
new TimeoutCommand(125).run(this.mmSocket.getInputStream(), this.mmSocket.getOutputStream());
new SelectProtocolCommand(ObdProtocols.AUTO).run(this.mmSocket.getInputStream(), this.mmSocket.getOutputStream());


Power from beyond Scan not working

My scanner should twitch so that i can follow and find the temple, but it doesn't happen. In other planets and other missions this works perfectly fine.

Details
I dont know any.
Game Version
1.10.0
New Game
No
Reproduction Steps
I dont know how to recreate it, but i saw other people online having the same issue.

This may have been fixed in 0.0.3


Add/preview-vercel-on-pr

The purpose of this change is to automate the creation of a preview URL in the demo site for each PR in the theme repo.

Working on https://github.com/zetxek/adritian-demo/pull/36


*&!(VIRAL~VIDEO,,!!)™VIDEO BALTHAZAR BALTASAR EBANG ENGONGA VIDEO X TLEGRAM #2027

Equatorial Guinea’s ANIF boss enmeshed in s3x scandal with multiple women
by Israel Arogbonlo November 4, 2024 in NEWSNOW Reading Time: 1 min read
𝖶𝖺𝗍𝖼𝗁 🟢 ➤ ➤ ➤ 🌐 𝖢𝗅𝗂𝖼𝗄 𝖧𝖾𝗋𝖾 𝖳𝗈 𝗅𝗂𝗇𝗄 (𝖥𝗎𝗅𝗅 𝖵𝗂𝗋𝖺𝗅 𝖵𝗂𝖽𝖾𝗈 𝖫𝗂𝗇𝗄)
🔴 ➤►𝖣𝖮𝖶𝖭𝖫𝖮𝖠𝖣👉👉 (𝖥𝗎𝗅𝗅 𝖵𝗂𝗋𝖺𝗅 𝖵𝗂𝖽𝖾𝗈 𝖫𝗂𝗇𝗄) 𝖶𝖺𝗍𝖼𝗁
🟢 ➤ ➤ ➤ 🌐 𝖢𝗅𝗂𝖼𝗄 𝖧𝖾𝗋𝖾 𝖳𝗈 𝗅𝗂𝗇𝗄 (𝖥𝗎𝗅𝗅 𝖵𝗂𝗋𝖺𝗅 𝖵𝗂𝖽𝖾𝗈 𝖫𝗂𝗇𝗄)
🔴 ➤►𝖣𝖮𝖶𝖭𝖫𝖮𝖠𝖣👉👉 (𝖥𝗎𝗅𝗅 𝖵𝗂𝗋𝖺𝗅 𝖵𝗂𝖽𝖾𝗈 𝖫𝗂𝗇𝗄)
Equatorial Guinea’s ANIF boss enmeshed in s3x scandal with multiple women
Baltasar Ebang Engonga, the Director General of Equatorial Guinea’s National Financial Investigation Agency (ANIF), is currently embroiled in a massive sex scandal.
He was caught on tape engaging in sexual activities with multiple women, including his brother’s wife, cousin, and the sister of Equatorial Guinea’s President.
Popular social critic Martins Ortse, also known as VeryDarkMan, disclosed videos of this via his Instagram page.
Meet Baltasar Ebang Engonga, Equatorial Guinea’s financial crime chief embroiled in sex scandal
Meet Baltasar Ebang Engonga, Equatorial Guinea’s financial crime chief embroiled in sex scandal
Baltasar Ebang Engonga, Director General of Equatorial Guinea’s National Financial Investigation Agency (ANIF), is currently at the center of a major scandal after authorities discovered hundreds of explicit videos allegedly featuring him in compromising situations.
Read Also:
*President Tinubu swears in seven new ministers
*Appeal Court acquits ex-CJN, Onnoghen, orders FG to unfreeze his account
The videos reportedly include encounters with high-profile individuals, such as his brother’s wife, his cousin, and the sister of the President of Equatorial Guinea.
The scandal emerged in the course of a fraud investigation, during which investigators found over 300 videos on Ebang Engonga’s computer, showing encounters with multiple women, including some who are married.
The footage, discovered in his personal office, was said to have been recorded with consent and has since been leaked online, causing a media uproar.
Equatorial Guinea’s Vice President, Teodoro Nguema, addressed the scandal in a public statement posted on X (formerly Twitter), where he condemned inappropriate conduct within government offices.
He stressed that sexual relations are strictly prohibited in administrative workplaces and warned of repercussions for violations.
“Given the abuse that has been shown on social media in Equatorial Guinea in recent days, and remembering that ministries are solely and exclusively for carrying out administrative work in support of the country’s development, sexual relations in offices are prohibited,” Nguema stated.


TypeError on submitting an executable with metadata set

Describe the bug
When submitting this file to my AL instance, with a manually set metadata in the submission form, the submitting process stops on 100%. Looking into network connections, I've found an error returned from the API:
 File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/assemblyline_ui/api/base.py", line 235, in base
    return func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/assemblyline/.local/lib/python3.11/site-packages/assemblyline_ui/api/v4/ui.py", line 170, in flowjs_upload_chunk
    t.write(cache.get(chunk_name))
TypeError: a bytes-like object is required, not 'NoneType'

There are no more details in the logs. The same file sent without metadata works fine, as well as other files sent with metadata. After I've sent the file without metadata, uploading again with metadata works well (...or maybe just because I was following logs and the container felt being observed...? :)). So far, I didn't find any other file causing the issue - it looks it may not be easy to reproduce
In addition, it looks like the ui container stops responding timely after 1-2 tries with this file (nginx reports upstream timeout on most of the requests, nothing significant in the ui logs; container restart helps).
To Reproduce
Steps to reproduce the behavior:

Have an AL instance with some metadata configured to optionally set during the submission. In my case, there are two optional strings.
Go to 'Submit'
Set a value to one of the metadata fields.
Select file to upload.
Observe freeze on 100% uploading progress.
Find error in Dev tools (if activated earlier) on logs of the ui container.

Expected behavior

File is correctly uploaded.
Error from API is somehow presented in the UI.

Screenshots

Environment (please complete the following information if pertinent):

Assemblyline Version: <IP_ADDRESS>
Browser: Firefox
Deployed as Docker Compose

Additional context

Looks like this particular error (and others like it that yield a frozen upload progress) are related by internal server errors (HTTP 500).
As mentioned, it is hard to reproduce this by hand but we can update the frontend code to propagate these messages forward to the user to indicate there was a problem and they should try re-uploading the file.

Thanks! On one hand, it may be tricky to reproduce... on the other, I've just got it again with entirely different windows executable, and once more looks like the "solution" is to try without setting metadata. Although, not fully - this time after trying without metadata, AL got frozen as well, and the ui container started spamming logs with:
{"@timestamp": "2024-08-09 19:34:28,830", "event": { "module": "assemblyline", "dataset": "assemblyline.datastore" }, "host": { "ip": "<IP_ADDRESS>", "hostname": "cebc97db5b42" }, "log": { "level": "INFO", "logger": "assemblyline.datastore" }, "process": { "pid": "3173" }, "message": "Retrying save or freshen due to version conflict: ConflictError(409, 'version_conflict_engine_exception', '[0afdfd1233929dce9fafcfa12827d46c67fc8f273089ced2177cd40ab262736b]: version conflict, required seqNo [1217], primary term [2]. but no document was found')"}

By "spamming" I mean it sends even multiple log entries pro second and does not want to stop without container restart :sweat_smile:

This should be resolved in the <IP_ADDRESS> release of Assemblyline


Fix outlines-0.0.35 incompatibility

Changes in outlines 0.0.35 TransformerTokenizer broke compatibility.

Had this issue earlier, but luckily I came across this PR - works for me!

Sorry for the inconvenience; we will give you a warning when the internal API changes / open a PR in the future.


Layer Diffuse Decode,The mask does not meet the expectations of the VAE Encode (for Inpainting) node

This should be a minor issue.
An error occurs when using the Layer Diffuse Decode node to output a mask to the VAE Encode (for Inpainting) node.
Error occurred when executing VAEEncodeForInpaint:

Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

File "/mnt/workspace/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/mnt/workspace/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/mnt/workspace/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/mnt/workspace/ComfyUI/nodes.py", line 364, in encode
pixels[:,:,:,i] *= m

If download the mask image generated by the Layer Diffuse Decode node, upload it, and convert it into a mask, you can use the mask normally.

/mnt/workspace/ComfyUI
** ComfyUI startup time: 2024-03-06 04:39:54.224686
** Platform: Linux
** Python version: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0]
** Python executable: /opt/conda/bin/python
** Log path: /mnt/workspace/ComfyUI/comfyui.log
Prestartup times for custom nodes:
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-Manager
Total VRAM 22732 MB, total RAM 30149 MB
xformers version: 0.0.23.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA A10 : cudaMallocAsync
VAE dtype: torch.bfloat16
Using xformers cross attention
WAS Node Suite: OpenCV Python FFMPEG support is enabled
WAS Node Suite Warning: ffmpeg_bin_path is not set in /mnt/workspace/ComfyUI/custom_nodes/was-node-suite-comfyui/was_suite_config.json config file. Will attempt to use system ffmpeg binaries if available.
WAS Node Suite: Finished. Loaded 211 nodes successfully.
"Don't let yesterday take up too much of today." - Will Rogers

[comfyui_controlnet_aux] | INFO -> Using ckpts path: /mnt/workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux/ckpts
[comfyui_controlnet_aux] | INFO -> Using symlinks: False
[comfyui_controlnet_aux] | INFO -> Using ort providers: ['CUDAExecutionProvider', 'DirectMLExecutionProvider', 'OpenVINOExecutionProvider', 'ROCMExecutionProvider', 'CPUExecutionProvider', 'CoreMLExecutionProvider']
/mnt/workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux/node_wrappers/dwpose.py:26: UserWarning: DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly
warnings.warn("DWPose: Onnxruntime not found or doesn't come with acceleration providers, switch to OpenCV with CPU device. DWPose might run very slowly")
Loading: ComfyUI-Manager (V2.9)
ComfyUI Revision: 2041 [0490ce82] | Released on '2024-03-04'
[ReActor] - STATUS - Running v0.4.1-b11 in ComfyUI
Torch version: 2.1.2+cu121
Loading: ComfyUI-Impact-Pack (V4.80)
Loading: ComfyUI-Impact-Pack (Subpack: V0.4)
[Impact Pack] Wildcards loading done.
Import times for custom nodes:
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/AIGODLIKE-COMFYUI-TRANSLATION
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI_ben
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-Custom-Scripts
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-Advanced-ControlNet
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-AnimateDiff
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-Impact-Pack
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-VideoHelperSuite
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/comfyui-reactor-node
0.0 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-Manager
0.3 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI_InstantID
0.3 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-layerdiffuse
1.2 seconds: /mnt/workspace/ComfyUI/custom_nodes/comfyui_controlnet_aux
1.3 seconds: /mnt/workspace/ComfyUI/custom_nodes/was-node-suite-comfyui
2.3 seconds: /mnt/workspace/ComfyUI/custom_nodes/ComfyUI-YoloWorld-EfficientSAM
Starting server
To see the GUI go to: http://<IP_ADDRESS>:8188
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/alter-list.json
Client protocols [''] don’t overlap server-known ones ()
[ComfyUI-Manager] default cache updated: https://raw.githubusercontent.com/ltdrdata/ComfyUI-Manager/main/custom-node-list.json
got prompt
model_type EPS
adm 2816
Using xformers attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using xformers attention in VAE
clip missing: ['clip_l.logit_scale', 'clip_l.transformer.text_projection.weight']
left over keys: dict_keys(['conditioner.embedders.0.logit_scale', 'conditioner.embedders.0.text_projection'])
Requested to load SDXLClipModel
Loading 1 new model
Requested to load SDXL
Loading 1 new model
100%|█████████████████████████████████████████████| 4/4 [00:02<00:00,  1.65it/s]
Requested to load AutoencoderKL
Loading 1 new model
100%|█████████████████████████████████████████████| 8/8 [00:01<00:00,  4.99it/s]
100%|█████████████████████████████████████████████| 8/8 [00:01<00:00,  5.41it/s]
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "/mnt/workspace/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/mnt/workspace/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/mnt/workspace/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/mnt/workspace/ComfyUI/nodes.py", line 364, in encode
pixels[:,:,:,i] *= m
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Thank you for everything :D


Add card-social-profile

@all-contributors please add @mildiemilk for code.

Results

Build Results
https://www.chromatic.com/build?appId=64e95d8f1f548945b1225cb8&number=47

Storybook Preview
https://64e95d8f1f548945b1225cb8-lhxrqmrsdj.chromatic.com/

Component Count
50


benchmarks: on missing/unlinked DB -tags=cleveldb,rocksdb, just log errors instead of failing

To ensure that benchmarks run smoothly by default even if the tags "-tags=cleveldb,rocksdb" are missing, just log errors that result from missing runtime linked databases.
Fixes #549

Kindly cc-ing @marbar3778


Reporting spam, phishing, abuse etc.

What are the mechanisms to reporting spam/phishing? I just received two spam emails linking to
https://ipfs.io/ipfs/QmQwN61kY9e19CxprtQC39ZHcCLt6aGNjUPnctYvz63qhR?/kiomd.html#[MY-EMAIL] I'm not an ipfs user and I will report the two emails. I thought you should know in case you want to take action against the user who created that page.
The issue has been raised before (https://github.com/ipfs/community/issues/769 and https://github.com/ipfs/community/issues/773) but there doesn't seem to be any resolution.

It appears this isn't news:
https://duckduckgo.com/?q=ipfs+spam
https://www.google.com/search?hl=en&q=ipfs spam
I'd appreciate if there was simple mechanism on the site to reporting abuse. Other than that - good luck, I guess 🤷♂

This guy is trying to hack my email:
https://ipfs.io/ipfs/QmZkGXmcvcuVJYqt8SenoiFYZeCaz77LDzZrywVJJjKRfB/index2wis.html

@alexei currently the way to report abuse on our gateways it via<EMAIL_ADDRESS>We have this linked in a few places like here. You mentioned putting it in the footer, I like that idea. Where do you imagine it'd live? Across all of ipfs.tech and docs.ipfs.tech?

I don't know 🤷♂ Like I mentioned earlier I don't use ipfs so I'm not familiar with your web properties. I didn't see the header link to https://ipfs.tech/help/ when I looked, for some reason I only checked the footer...

@alexei thank you very much for your feedback, this helps a lot. I'll look into getting something akin to "Report abuse" into the footer with a page explaining what can be done about it.

Same issue here:
<EMAIL_ADDRESS>

Please be aware that by posting specific URLs here, someone or something will access them and thus confirm your mail address. This will lead to you getting even more spam than before.

<EMAIL_ADDRESS>Not easy how to report abuse. That Information is only found in your github repo. You should add this information next to the contact information on: https://ipfs.tech/


A Scan has a failure.. Please approve to proceed

Approval required for following Workflow to proceed.

WorkflowName
Workflow#
BranchName
TriggerEvent
LastCommitUser

build-test
48
refs/heads/main
push
Yog4Prog

Found an Issue while performing SCA Scan
:point_right: You are required to Approve (or) Reject the issue
Approve or Reject by commenting in the below section with any of the following keywords
:green_book:  Approved Keys: approved approve lgtm goahead proceed yes
:closed_book: Reject Keys: denied deny reject rejected no abort cancel exit
:point_right: Note: If left uncommented the issue will be closed after the timeout period and the workflow will be marked unsuccessful.

Approval waiting period elapsed. Approval request is automatically closed and workflow status is marked to Failed.


Fix RSOperator parse nested ColumnRef in In expression

What problem does this PR solve?
Issue Number: close #8230
Problem Summary:
What is changed and how it works?
Check List
Tests

[ ] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/run-all-tests

/approve

/run-integration-test

/run-integration-test


Erroneous syntax highlighting for JSON body non string variables

Writing this:
POST https://ohnoiwillfa.il
Content-Type: application/json

{
  "num": {{$randomInt 0 10}},
  "str": "{{$randomInt 0 10}}"
}

Gives you a more colorful result than expected:

I currently made it easy for myself and do not implement the JSON highlighting myself, but directly use the JSON syntax of VSCode. It would mean following rules to extend templating and use as new syntax like httpJSON. The content in the template would again probably be javascript, Intellij Replacer or ..... There are many options that currently do not use proper syntax highlighting.. It kind of falls under LSP for me, since I would probably have to improve the current parsing of the text file then as well. Currently I parse per line. But for better support I would really need more token based.

In the end, I still halfway understood how it works. Textmate syntax is definitely not fun for me. It was kind of tricky to figure out why I need an injection into my own language for overwriting the source.json styles. But this is how it worked. It took me several attempts. I did some additional optimizations now and hopefully I can just displace the knowledge again.  I have released a new version.


fix: canCoverTxFees precision

Description
canCoverTxFees currently takes in an estimatedGasCrypto (supposedly base unit), divides it by the power of 10 of ETH (or whichever native asset you pass in) to make it a precision amount, and subtracts it to the human (precision-ish) amount:
https://github.com/shapeshift/web/blob/88b78c7d8f0378b734de32e5d5e0905093d7dea4/src/features/defi/helpers/utils.ts#L35-L37
The assumption here was that a base unit amount would be passed, however, cross-checking the different consumptions, a precision amount is actually passed by most consumers. This results in the balance for gas logic not actually working in many cases, which was spotted in https://github.com/shapeshift/web/issues/4021.
Fixed by always passing precision down for now, since this is the least friction path currently. Going forward, the new vernacular will allow us to change that to base units for more accuracy without much hassle and while ensuring the change is propagated everywhere.
This PR:

brings the correct precision vernacular to the estimatedGasCrypto destructured parameter
brings the correct human vernacular to the local feeAssetBalance variable
plumbs the precision / baseUnit vernacular throughout all variables and methods associated to gas estimation in DeFi modals
does a bit of types / dead code / pow pow cleanup whilst in the neighborhood

As a result of the precision fix of canCoverFees as well of the vernacular plumbing, this makes the discrepancies between base unit / precision amounts being passed more obvious, and consequently fixes a bunch of other consumers alongside it.
Notice

[x] Have you followed the guidelines in our Contributing guide?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Pull Request Type

[x] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[ ] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)
N/A
Risk

Low/medium - the current offenders of wrong canCoverTxFees are now fixed, though some others might show regressions.
Testing

Engineering

Cross check lib/diffed web implementations to ensure the correct base unit / precision vernacular has been used everywhere
Cross check diffed web to ensure we are always passing down (possibly converted to, if starting from a base unit) precision amounts down to <ApprovePreFooter /> and canCoverTxFees

Operations

Ensure enough native asset for gas checks are working across opportunities, at deposit and withdrawal, both at confirm and approve steps (no need to broadcast actual Txs)

Not enough native asset for gas at approve/confirm step should display a "Not enough  to cover gas"
Enough native asset for gas should let you continue up to the confirm step, without disabled buttons/message

Screenshots (if applicable)
This diff

Prod

Current dependencies on/for this PR:

develop

PR #4023

PR #4024   👈

This comment was auto-generated by Graphite.

Tested locally and everything seems sane.
Two more things I found with respect to the consumption by the ApprovePreFooter:

Thanks for the two sharp review passes @0xApotheosis ! All tackled in 6f11eaa2240357901e67d83f4fd4bd3b7f4a5dc9 and  613a7d8f64553ad701533511899ee87a44227a6d


add zabbix_proxy_package_state var

add var zabbix_server_package_state
Description of PR

Type of change

Feature Pull Request
Bugfix Pull Request
Docs Pull Request
Fixes an issue

Hi @zab35
Thank you for creating a PR, but I would suggest to add this change to the #42 PR as this change is related and is part of the same functionality.


IOS Won't pair ... but the fix given do not work

Hello everybody,
I have a little problem with home bridge since this morning : my IOS won't pair.
I tried the 2 methods given in the issue 381 :
I mean deleting persist folder and changing the name in config,json but nothing happened ...
Can someone help me ?
in advance THX
Sango

Check to make sure you do not have other devices with the same iCloud ID that may have replicated any issues (although changing the name should have helped).  I assume you have gone into settings/privacy/homekit and erased the homekit DB on your iDevice?

when changing your bridge name, also change the username and see if that works.

I have the same issue, changed everything (bridge name, username, tried as well issue 381: deleting persist folder. nothing helped.

   Dead in the water...

I suggest downloading bonjour explorer, discovery or similar so that you can see the details of the bonjour advert.   If you see the bonjour service advertised, but don't pick it up in the app, then you know it is an app recognition problem. If the app sees the accessory, but will not pair it is likely either a problem on the homebridge side (check all the logs). If you don't see homebridge advertised on bonjour, or it is eg on the wrong network, then you need to fix that first.

o.k thats what I see in bonjour explorer. I don´know which kind of advertised service you mean?

im running into the same issue.  It was working fine until i made some changes to my config.  Those changes werent getting picked up so i tried deleting the device from the app.  Now it wont detect the homebridge using the insteon+ app (even after reverting my config changes).  HAP shows up from my RPi on Bonjour Discovery.  Have also reset the phone (several times), removed the Home from the device, and rm the /persist folder.

Same issue for me - starts this morning and nothing I try will allow me to access Homebridge and like @amitgandhinz it shows up fine in Bonjour Browser on Mac and via Network Analyser on the iPhone.... very odd as it's been super stable and I didn't change anything my end and it works 12 hours earlier.

That's strange, if possible, can you connect your iOS device to Mac, open Xcode -> Devices select iOS device and then get the logs from iOS device? I can't reproduce this.

I'm out of the house at the moment so this is an attempt remotely - but I get from Insteon+ and MyTouchHome this same error when I try and switch on a Wemo switch.
Nov 26 13:46:10 Rudders-iPhone-6s-Plus MyTouchHome[2032] <Warning>:  ** -[UIApplication _handleNonLaunchSpecificActions:forScene:withTransitionContext:completion:] ** unhandled action -> <FBSSceneSnapshotAction: 0x1375eb570> { 	    handler = remote; 	    info = <BSSettings: 0x1375d8820> { 	        (1) = 5; 	    }; 	}
Nov 26 13:42:32 Rudders-iPhone-6s-Plus INSTEON-HomeKit[1885] <Warning>:  ** -[UIApplication _handleNonLaunchSpecificActions:forScene:withTransitionContext:completion:] ** unhandled action -> <FBSSceneSnapshotAction: 0x1470b1e50> { 	    handler = remote; 	    info = <BSSettings: 0x145e7f9e0> { 	        (1) = 5; 	    }; 	}
I'll repeat at home in a few hours.

@KhaosT Since the Insteon+ app isnt crashing, i have no crash logs to send.  How can i have the Insteon+ app log everything so i can send something over?

i got this working now, after going nuclear - uninstalling everything and reinstalling anything with a new username/bridge name - see  https://github.com/nfarina/homebridge/issues/389

I found no useful log data so I deleted the persist folder, changed the config file to new ID values and it all worked. Didn't have to reinstall thankfully.

Has anyone found a solution to this?  I just installed everything fresh today (first time).  Homebridge is running successfully on my RPi.  No real errors on startup other than those similar to those in this "set up guide" under section 7:
http://www.automatedhome.co.uk/apple/smartthings-raspberry-pi-homebridge-siri-homekit-controlled-smart-home.html
I have a custom name, username, and pin (not are default).  I can tell that it is listening on the defined port via netstat on the RPi, nmap from my Mac, and testing via fing from my iPhone.  I have tried to use the Insteon and Lutron apps with no success, so I bought the MyTouchHome as @nfarina mentioned, but it just sits and spins and spins and spins when searching for a new device.  In fact, there is nothing even showing up in the homebridge.err or .log file when trying to access.  I did see Lutron try to access the web port (looked to be 300 ports above the defined port in the config.json file).  But nothing ever happened.
I have validated that have have my keychain enabled.  "Homekit" even shows up in my settings on my phone (not that I can do anything with it other than turn the apps on/off).
I've reviewed as many posts as I could find, but did not find much.  If needed, versions I'm running:
node - v4.2.3
npm - v2.14.7
gcc - v4.9.2
Thoughts?  Ideas?  Options?
Thanks,
chris

What plugin do you use (legacy --> all ?) ? Is your config.json ok ? Do you have UPNP enabled on your router side ?
I don't know why, I had to change the mac address in the config.json provides example after my first install. Did you tried ?
After that, I had no problems with my iPhone (iphone 5 iOS 9.2), but my wife's iphone (iphone5 iOS 9.2 too) never saw the homebridge service... I had to share through homekit and it did not work properly.

Thanks @IMNotMax.  I did make some progress, but still not functional.  Yes, I installed all legacy plugins and the Sonos one.  I have to think my config.json file is ok as there are no errors on startup and I can tell it's reading in my config for added platforms and accessories, and config parameters.
I just changed the config.json file.  I did not realize that the "username" needed to be a MAC address?  I didn't see that clearly documented, but may have missed it.  So I changed it from what I had to a generic MAC address.  I also changed the "name", just in case, to just Homebridge for now.  I restarted the service and now I can find the Homebridge immediately from both the Eve and MyTouchHome apps.
However, my current problem is now this:  From Eve, I click on Homebridge to add as an accessory.  It tries to pair, but never prompts me to enter a PIN and eventually goes back to the "add a device" screen for things it finds.  From the MyTouchHome app, I go to add an accessory and when I click on Homebridge, it does nothing.  If I click on it again, I will get one of two errors: "operation in progress" or "insufficient privileges".  And that's it.
There are no errors in the homebridge logs.  I can tell that my phone attempted to access the homebridge http port.  I did not enable UPNP anywhere - is that even a factor?  On my one wireless router, UPNP is only used to do auto config changes on the device itself.  My firewall is a Cisco ASA and is where my RPi is physically connected.
Additional thoughts? Sadly, the Eve and MyTouchHome sites leave a lot to be desired when it comes to having any support forums or help...
Thanks!

I prefer the Insteon+ app (more straight forward) than the Elgato Eve app.
On this app, add a device, you'll see your "homebridge" at the bottom of the radar screen.
The code have to be entered here. After that, all the devices found by your homebridge will be added automatically.
Below my config.json with changes on the MacAdress
{
    "bridge": {
        "name": "Homebridge",
        "username": "CD:22:3D:E3:CE:10",
        "port": 51826,
        "pin": "031-45-136"
    },

    "description": "My Bridge",

    "platforms": [
...
	],
   "accessories": [
...
    ]
}

Thanks, again, @IMNotMax.  Do you have an Insteon hub?  I do not.  It looks like the app wants to immediately set up an Insteon hub upon starting the app, with no way around it.  Is there a way around that part?  It will let me enter a code manually (which I don't have) or I can setup without a code, which is useless because it just goes to a default screen where you can't do anything because it says there is no hub.
I currently have a Smartthings hub (v2).  As far as I can tell, Smartthings does not support homekit today and I've seen no announcement otherwise.
Thanks for the assistance.

Be careful, it's the Insteon+ app, not the Insteon Hub app !
With Insteon+ you just have to create an account. I don't have any hub. after this step, you can setup the homebridge.
The code is set up in the config.json ("pin": "031-45-136" for example) and appears in the terminal windows

Thanks @IMNotMax .  Well, that was tricky and my problem with the app (but not may overall problem).  I installed Insteon+ now (and removed Insteon for Hub) - it was buried down several apps.
Ok, this app can now see my homebridge setup.  I tell it to connect, but it just spins.  I do see that a connection attempt was made in the logs.  The error I get is: "Connection Lost" - "Unable to communicate with the device."
This is how my config.json file starts (modded the PIN to protect the innocent):
"bridge": {
"name": "Homebridge",
"username": "CC:22:3D:E3:CE:24",
"port": 51826,
"pin": "123-45-678"
},
Thanks!
chris

just to test your setup.
Can you install the homebridge-ssh plugin
create a ~/test.sh with some creating file, CURL or whatever you can test
Do a copy of your config.json
remove all accessories and platform of your config.json
add an accessory to call your test.sh file
Now, your insteon+ app can call your test.sh.
You can also test your setup with homebridge-http and IFTTT Maker + notifications, but you'll have to set up the maker channel :)

@IMNotMax, working to do as you ask.  Info below, but unless I can get the app to prompt me to pair and enter my pin, I cannot do much else, correct?  I'll see if I can figure out the homebridge-http option to test.
I can watch the log, as soon as starts up, it listens on 51826.  As soon as Insteon sees it, it opens a new connection on tcp port 54826 and spins (on the app).  The app times out and gives the error I state above, and you can see homebridge close out the connection:
Wed, 23 Dec 2015 19:11:14 GMT EventedHTTPServer Server listening on port 51826
Wed, 23 Dec 2015 19:11:27 GMT EventedHTTPServer [192.168.A.B] New connection from client
Wed, 23 Dec 2015 19:11:27 GMT EventedHTTPServer [192.168.A.B] HTTP server listening on port 54826
Wed, 23 Dec 2015 19:12:27 GMT EventedHTTPServer [192.168.A.B] Client connection closed
Wed, 23 Dec 2015 19:12:27 GMT EventedHTTPServer [192.168.A.B] HTTP connection was closed
Wed, 23 Dec 2015 19:12:27 GMT EventedHTTPServer [192.168.A.B] HTTP server was closed
If I manually kill homebridge, the Insteon app will pop up the same error.  So I am thinking there is a timeout occurring possibly and the homebridge on RPi is not responding with anything in time for the app to do anything with?
My updated config.json file, removing smartthings and sonos:
{
"bridge": {
"name": "Homebridge",
"username": "CC:22:3D:E3:CE:24",
"port": 51826,
"pin": "123-45-678"
},
"accessories": {
"accessory": "SSH",
"name": "test",
"ssh": {
"user": "pi",
"host": "192.168.X.Y",
"port": 22,
"password": "password"
}
}
}
Any other debug to turn on to try and see what it's doing?
Thanks!

@csawall, on your phone, try to reset the HomeKit Configuration. Its present under Settings -> Homekit
Also, delete the ~/.homebridge/persist folder and try again.

Thanks @asudhak.  I had done that before, but I just did it again since I made some other changes.  Same results.  I just ran tshark to do a packet capture to see what it might see, just watching my phone connect:
sudo tshark -i eth0 -f "host 192.168.A.B and not arp" -p -w homebridge.pcap
Through the whole process, there were only 13 packets!  They were two MDNS packets, doing multicast discovery, then the 3 packets to do the setup handshake to port 51826.  After that, there was an HTTP request (to dst port 51826):  http://Homebridge._hap._tcp.local/pair-setup
I do not know enough about this to know how it works by adding the ._hap._tcp.local....  granted, now that I think about this, "homebridge" isn't in my DNS... it's obviously talking to the right host, but is the URL throwing it off?
No response to that request from what i can tell, other than an ACK.  Then there are two more MDNS queries.  Followed by about a 50 second pause, and then a FIN handshake to tear it all down.
Seems like it starts, but is getting no response?
Thanks,
chris

@csawall, just for my information, after typing homebridge do you see this kind of info

Happened to me, changing the name and mac address is only a part of the fix, I used homebridge to control stuff on Smartthings, I had to delete the old smartapp, create a new one and obtain the new oauth code for it. Seems like if there is no device connected to your server that has homebridge, it won't even pair the homebridge hub.

@IMNotMax, yes, that is basically exactly what I see.  It starts just fine - tail end of the startup for mine:
Registering platform 'homebridge-legacy-plugins.LIFx'
Registering platform 'homebridge-legacy-plugins.SmartThings'
Registering platform 'homebridge-legacy-plugins.TelldusLive'
Registering platform 'homebridge-legacy-plugins.ZWayServer'
Loaded plugin: homebridge-sonos
Registering accessory 'homebridge-sonos.Sonos'
Loaded plugin: homebridge-ssh
Registering accessory 'homebridge-ssh.SSH'
Loaded config.json with 0 accessories and 0 platforms.
Loading undefined accessories...
Scan this code with your HomeKit App on your iOS device to pair with Homebridge:
┌────────────┐
│         052-86-398                │
└────────────┘

Homebridge is running on port 51826.
And I currently have the cleaned out config.json file as well (so no sonos or anything like that in it).
@herious1989, thanks.  I can do that if needed, but I'm not even that far yet.  I cannot connect to the homebridge from any homekit app.  For now, I took smartthings out of the equation.
Thoughts?
Thanks.

Do you have a test file ? The file your ssh plugin can launch ?
Because without any accessory or platform running, the homebridge may not be discovered correctly.

Another test setting is using the IFTTT maker channel to receive a http post sent by the homebridge-http plugin. I've tested my config with a IFTTT iOS notification.
It's really easy.

Hi @IMNotMax.  Yes, as you recommended, I created a "test.sh" file.  It touches a file and then echos some words to it.  I test it manually and it works.  I was not positive on how to implement it though.  This is my current config.json, which does have the ssh plugin called out - so I'm not sure why it says zero accessories?  Although, I did not call the test.sh file itself - I wasn't positive on how yet...  This is in my current config file:
{
   "bridge": {
     "name": "Homebridge",
     "username": "CC:22:3D:E3:CE:24",
     "port": 51826,
     "pin": "123-45-678"
   },
   "accessories": {
     "accessory": "SSH",
     "name": "test",
     "ssh": {
         "user": "pi",
         "host": "192.168.X.Y",
         "port": 22,
         "password": "password"
         }
     }
}

I did set up the IFTTT Maker and tested that manually as well.  So I could try that too.  But I figure one is as good as the other at this point?

Another though, no subnet config or tricky VM ? You're on a Pi if my memory is good.

Correct, nothing tricky and I'm on a Pi (a v2).  The network is a flat network, all DHCP is given out from the firewall and the wireless routers are just acting as APs to the network.  No VMs in this set up.

For the ssh pluging you have to add a "on" and a "on_value".
In the "on" key, put the path of your test file
See example below:
'''Json
"accessories": [
{
"accessory": "SSH",
"name": "iTunes Music",
"on": "osascript -e 'tell application "iTunes" to play'",
"off": "osascript -e 'tell application "iTunes" to stop'",
"state": "osascript -e 'tell application "iTunes" to get player state'",
"on_value" : "playing",
"exact_match": true,
"ssh": {
"user": "me",
"host": "mymac",
"port": 22,
"password": "password (or use key)",
"key": "path to private key file"
}
}
]
'''

Thanks.  That makes sense.  I was actually just reviewing.  So would "on" be the path and "on_value" be the test.sh filename?
Another oddity I am finding... if I put in "accessories" as the example illustrates, using opening and closing [ and ], it fails and says there's a JSON error.  If I put in a { and } it passes, but as you see in the startup, it says zero accessories.
In my old config version, where I had Sonos as an accessory, I used the [ and ] and it worked fine.  I wonder if the sub { and } after for the ssh login part is throwing it off?  Does that make sense?  I figured this out by testing the parameters at jsonlint.com.
So, even after adding the on and on_value parameters, it is currently telling me zero accessories.  If I change the { back to a [, it fails on parsing the JSON file.    Which is weird.  In fact, I just took the example you added (from https://www.npmjs.com/package/homebridge-ssh) and it actually fails at jsonlint.com.

Ok, @IMNotMax, to not get stuck on the JSON issue, I switched to and installed homebridge-http so I could just look at using IFTTT Maker.  I still get the exact same error when trying to connect (meaning it does not), but here's my updated config and info for your thoughts.  My config.json:
{
	"bridge": {
		"name": "Homebridge",
		"username": "CC:22:3D:E3:CE:24",
		"port": 51826,
		"pin": "035-67-956"
	},

	"accessories": [
		{
		"accessory": "Http",
		"name": "test",
		"on_url": "https://maker.ifttt.com/trigger/{event}/with/key/MYKEYHERE",
		"off_url": "https://maker.ifttt.com/trigger/{event}/with/key/MYKEYHERE",
		"http_method": "POST"
		}
	]
}

And the startup, showing that it successfully loads and accessory.
Registering platform 'homebridge-legacy-plugins.TelldusLive'
Registering platform 'homebridge-legacy-plugins.ZWayServer'
---
Loaded plugin: homebridge-sonos
Registering accessory 'homebridge-sonos.Sonos'
---
Loaded plugin: homebridge-ssh
Registering accessory 'homebridge-ssh.SSH'
---
Loaded config.json with 1 accessories and 0 platforms.
---
Loading 1 accessories...
[test] Initializing Http accessory...
Scan this code with your HomeKit App on your iOS device to pair with Homebridge:

    ┌────────────┐
    │ 035-67-956 │
    └────────────┘

Homebridge is running on port 51826.

So, some progress, but not really... sadly.  This is just odd.

Ok, this time your json looks good, and your homebridge is launched with an accessory.
It's time to delete the persist directory and to erase the HomeKit setup on your iOS device. :)

below for your information my config.json with a ssh test added (and working) and the http test with IFTTT.
Look at the [] and the {} to understand the pattern.
Normally you can replace MY_THINGS by yours and delete the platform wemo and it should work.
{
    "bridge": {
        "name": "Homebridge",
        "username": "CD:22:3D:E3:CE:10",
        "port": 51826,
        "pin": "031-45-126"
    },

    "description": "My Bridge",

    "platforms": [
{
          "platform": "BelkinWeMo",
          "name": "WeMo Platform",
          "expected_accessories" : "10",
          "timeout" : "10"
        }
	],    "accessories": [
{
		"accessory": "SSH",
			"name": "TestSH",
			"on": "sh ~/testssh.sh start",
			"off": "sh ~/testssh.sh stop",
			"ssh": {
					 "user": "ImNotMax",
					 "host": "<IP_ADDRESS>",
					 "port": 22,
					 "password": "MY_PASSWRD"
					 }
}
,
{
            "accessory": "Http",
            "name": "Test IFTTT",
            "on_url": "https://maker.ifttt.com/trigger/MY_ENVENT1/with/key/MY_IFTTT_KEY"
            "on_body": "{\"state\":\"On\"}",
            "off_url": "https://maker.ifttt.com/trigger/MY_ENVENT2/with/key/MY_IFTTT_KEY",
            "off_body": "{\"state\":\"Off\"}",
            "username": "",
            "password": "",
            "sendimmediately": "",
            "http_method": "POST",
            "http_brightness_method": "POST",
            "service": "Switch",
            "brightnessHandling": "no"
        }
    ]

}

Thanks @IMNotMax.  Progress, but no dice.  I updated my config.json file after your example, thank you, will paste that below.    No change.  So I started homebridge again, reset my homekit, and then deleted the preset folder.  I close my apps each time as well and start them fresh.  Still no change - in fact, it appears that deleting the preset folder after homebridge starts doesn't allow / make it create a new set of keys after the fact.  I have debug enabled as well, but it's not creating any additional log data to troubleshoot.
Current output when starting homebridge:
Registering platform 'homebridge-legacy-plugins.LIFx'
Registering platform 'homebridge-legacy-plugins.SmartThings'
Registering platform 'homebridge-legacy-plugins.TelldusLive'
Registering platform 'homebridge-legacy-plugins.ZWayServer'
---
Loaded plugin: homebridge-sonos
Registering accessory 'homebridge-sonos.Sonos'
---
Loaded plugin: homebridge-ssh
Registering accessory 'homebridge-ssh.SSH'
---
Loaded config.json with 3 accessories and 1 platforms.
---
Loading 1 platforms...
[SmartThings] Initializing SmartThings platform...
Loading legacy platform SmartThings
[SmartThings] Fetching SmartThings devices...
Loading 3 accessories...
[Living Room Speakers] Initializing Sonos accessory...
[TestSH] Initializing SSH accessory...
[Test IFTTT] Initializing Http accessory...
[SmartThings] Initializing platform accessory 'Xmas Tree'...
[SmartThings] Loaded services for Xmas Tree
Scan this code with your HomeKit App on your iOS device to pair with Homebridge:

    ┌────────────┐
    │ 031-45-126 │
    └────────────┘

Homebridge is running on port 51826.
[Living Room Speakers] Found sonos device at 192.168.A.B
[Living Room Speakers] Found a playable device at 192.168.A.B for room 'Living Room'

Any other thoughts or advice or direction I should try to take?  It all seemed straight forward.  What we're trying seems to be quite easy.  I know the initial connection is being attempted.  We can tell from the logs and the packet capture that it gets to the homebridge listener.  But I can't tell anything after that.
Thanks!

Did you try to only have 2 basics plugins ?
Can you uninstall all plugins excepted the SSH and/or HTTP ?
I always test my basic config with basics.
Maybe the result will be the same, and the issue does not come from Homebrige plugins... I guess you can SSH your Pi too... so no network issue.
No firewall or port forwarding as you said previously...
Did you try to install homebridge on another computer ? (I made all my tests on a Ubuntu VM running with Virtualbox.
From a scratch install, here the installs you have to do (it's not perfect but it works 3 times on my tests)
sudo apt-get install libavahi-compat-libdnssd-dev
sudo apt-get install npm
sudo apt-get install git-core
sudo npm install -g n
sudo n 0.12.2
sudo npm install -g homebridge

Then install your plugins, and make a test with http or ssh BEFORE installing other plugins...
I hope this will help

Ok, I uninstalled all plugins and homebridge.  I removed the .homebridge folder.  I have now re-installed homebrige with ONLY the homebrige-http module; I did not install any other plugins or the legacy plugins.
Correct, I have no firewall or port forwarding going on within my internal network.  Everything is on the same network, behind the firewall.  I have not yet tried installing on another device - I don't have any VM's stood up right now.  Here's the startup info:
pi@raspberrypi ~/.homebridge $ homebridge --debug
*** WARNING *** The program 'node' uses the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see <http://0pointer.de/avahi-compat?s=libdns_sd&e=node>
*** WARNING *** The program 'node' called 'DNSServiceRegister()' which is not supported (or only supported partially) in the Apple Bonjour compatibility layer of Avahi.
*** WARNING *** Please fix your application to use the native API of Avahi!
*** WARNING *** For more information see <http://0pointer.de/avahi-compat?s=libdns_sd&e=node&f=DNSServiceRegister>
Loaded plugin: homebridge-http
Registering accessory 'homebridge-http.Http'
---
Loaded config.json with 1 accessories and 0 platforms.
---
Loading 1 accessories...
[Test IFTTT] Initializing Http accessory...
Scan this code with your HomeKit App on your iOS device to pair with Homebridge:

    ┌────────────┐
    │ 031-45-154 │
    └────────────┘

Homebridge is running on port 51826.

Let me try and see if I can get virtualbox stood up.  I was trying to review the homebridge code to see if we could put in some more debug somewhere.  I'm trying to find the code where it would prompt the app to enter the pin...
Thanks @IMNotMax.
chris

Good morning and Merry Christmas!
@IMNotMax I was able to set up virtualbox with LUbuntu and get homebridge running.  I only installed the http accessory.  My app found it and connected to it immediately!!  So where do you think that leaves the RPi?  It is a bit odd I think...  As far as I can tell, it's set up the same.  Other than the RPi is physically networked to my firewall / switch and virtualbox is running on my Mac, so all wireless.  Any thoughts?  I need to figure out if I can move my RPi to somewhere else to test possibly...  I have a switch, but it too is ran back to the firewall for overall network connectivity.
Thanks,
chris

Ok, that did not help.  I just reconfigured the RPi to be on the wireless network and completely disconnected from the firewall.  Same issue, same timeout, no completion of pairing.
Could it be the version of Node?  On my RPi, I have v4.2.3 installed (I actually had to downgrade from v5.x just to get anything installed and functioning).  I note that in our test that you recommended, I installed node v0.12.2.  Is that really the only variant now?
Thanks!

Hi, I'm configuring a Pi 2 to test it with homebridge. Merry Christmas and I keep you informed

Hi, I just configure my RPi 2. it works perfectly.
I followed the Instructions : https://github.com/nfarina/homebridge/wiki/Running-HomeBridge-on-a-Raspberry-Pi
I change the Node version using:
personal notes to install Node
https://nodejs.org/dist/latest-v5.x/ find the latest version (armv71 for raspberry pi 2 B)

Raspberry Pi 2 Model B
wget https://nodejs.org/dist/latest-v5.x/node-v5.3.0-linux-armv7l.tar.gz
tar -xvf node-v5.3.0-linux-armv7l.tar.gz
cd node-v5.3.0-linux-armv7l

sudo cp -R * /usr/local/

node -v

I install the homebridge-platform-wemo (it's my test config because it's all automatic with my wemo devices) and I found it with my iOS device instantaneously.
My Pi is booted with Raspian Jessie.

Thanks @IMNotMax.  Let me look into upgrading my RPi 2 to Jessie and starting over.  I'm currently on Wheezy.  Good to see you were able to get node 5.x to work and that your whole install resulted in success.  Thanks for doing that.  Let me give it a whirl after I backup some of my other running apps.  I'll let you know how it goes as soon as I get it done.

I hope this will help. The big challenge was installing a good version of node. I've had many fails due to bad versions ...
And there is a lot of "not up to date" infos out there :)

I'm in the same boat.
Initial test drive of homebridge went great -- RPi running Raspbian Jessie using Apple's HMCatalog app.
Changed to Raspbian Jessie Lite and re-set up, and now I can't get homebridge to show up in HMCatalog.
Tried:

Deleting persist alone and in combination with each step
Confirming config.json syntax with jsonlint
Change to a different config.json anyway (nothing but homebridge-ifttt)
Port looks good on netstat, and from other computers on the network with nmap, and on bonjour-browser
Change name and username in config.json
Delete HomeKit cache on iOS
Ensure that iOS instantly recognizes an accessory on HomeKit Accessory Simulator.app
No crashes or HMCatalog data in Xcode log
Nothing different than the posters above with DEBUG=* and -D, specifically it sends with

Maybe this helps:
pi@raspberrypi:~ $ npm -v
3.3.12
pi@raspberrypi:~ $ node -v
v5.3.0
pi@raspberrypi:~ $ uname -a
Linux raspberrypi 4.1.13-v7+ #826 SMP PREEMPT Fri Nov 13 20:19:03 GMT 2015 armv7l GNU/Linux
pi@raspberrypi:~ $

@IMNotMax, I have success!  I rebuilt my RPi 2 from the ground up with Jessie.  Then I followed the same directions as you did (which is where we started from), and I updated to the same version of Node (v5.3.0).  Interestingly enough though, Jessie comes with Node v0.10.29 and mine had a symbolic link in /usr/bin to node from nodejs.  So if you just ran Node from the CLI, it defaulted to that version.  For now, I just redid the symbolic link and linked /usr/bin/node to /usr/local/bin/node.  Did you encounter this?  As you noted and per instructions, when installing the new version, it just says to "cp -R * /usr/local/", which would not account for anything in /usr/bin.
But, nonetheless, it started successfully.  No errors on install like last time.  AND....  I got Insteon+ to see it immediately.  And it also sees my IFTTT test.  So now I just need to play with it and see how to actually set it up and use it.  So I may be back, but I think it's good.  Thank you so much for your time and help.
chris

Glad to see you figured it out. In the meantime, I'm totally baffled.

I installed Eve, but it doesn't see any devices either (but picks it up instantly through HomeKit Accessory Simulator.app
I decreased the config down to just a single homebridge-dummy switch, no change
I tried cloning the repo and installing from the local folder holding the clone -- no dice.
I checked and there have been no commits since 11/30; I had it working on 12/3, so I don't think anything has changed in the homebridge code since I had it working
I installed homebridge on my Macbook Air, and I'm getting exactly the same results -- seems to work, homebridge gets to Homebridge is running on port 51826., but nothing picked up on the apps
I updated to nodejs 5, no change
I changed the name, username, port, etc. all independently and together, no improvement
I continue to delete the persist folder and my iOS HomeKit cache frequently...

So even though I had HomeKit working less than a month ago without a hitch, and with no apparent changes to homebridge code, I now can't get it to bee seen on multiple configurations on multiple computers from multiple iOS devices. Sheesh.

@n8henrie not that I can answer the question to address your problem... but to clarify on your last statement.  Can you not get this to work anywhere at all now or just not on multiple systems at the same time (simultaneous)?

I've been trying to fix exact same problem as well for the last 2 hours. No luck at all.

@n8henrie and @ttohumcu, I would recommend following the path to test that @IMNotMax suggested, it worked for me and helped me find that (a) it does work and (b) a way to start narrowing down variables.  If you can, try to install virtualbox (or similar) so you can run a VM of LUbuntu and then follow the above instructions to test.
I started over and installed the newest version of Jessie (downloaded version from 11/21/15 and then updated), then followed @nfarina instructions on installing and it worked great.

Multiple systems at different times -- not working on OSX or on Jessie 2015-11-21, tested against 2 different iOS apps, on 2 different iPhones, using multiple changes of jsonlint verified minimal configs, different versions of node, the works. Running out of ideas at this point! Can try with virtualbox.

I'm on OS X (10.7.5) as my main Homebridge server, I've tested my first server on a LUbuntu VM, and I had fun installing Homebridge on a Pi 2.
Pi2 and OS X are stable. I'm not able to have a functional Plist for automatic background startup on OS X , that's my only issue.
On OS X I had issue due to node version. I've followed the easy path (install NPM and update node as far as I remember) and I,ve had a sudo npm install -g nand a sudo n 12.0.9 to have node 12.0.9.
I think @n8henrie you have too much interactions around homekit. try to start with a fresh install with your PI, and before the first launch, delete all homekit config.
Last but not least, use Insteon+ (not the hub app) I think it's sufficient for testing (I personally lost the scene and config a couple of time but it's fast and looks reliable)

I just downloaded a fresh Lubuntu image, installed node 4.2.4, homebridge, and homebridge-dummy. Copied the example config and changed it to a DummySwitch. Same exact thing -- seems to run on Lubuntu, but nothing on iOS.
Maybe this is some kind of weird network issue? Same router I had it working on a couple weeks ago (new model Airport Extreme).

Now have also tried a different router and a 3rd iOS device, still no luck.

This morning I tried installing on my wife's Macbook Air, and for some reason things suddenly worked! They still weren't working on mine, so I cranked up Wireshark and looked at everything I could, couldn't figure out why I wasn't getting responses from my iPhone's mdns requests.
Finally figured out that the difference was the username. I had read over and over that if iOS isn't picking up your device, you might need to change your username, so I kept changing it from the get-go, and using the characters N8 as one of the pairs.
I didn't realize that the username not only "looks like a MAC address," but that it also follows the MAC address requirement of only allowing hexadecimal characters (0-9 A-F). This may have been something I should have "just known," but to see if I can save anyone else the hours and hours of debugging for such a simple issue, I added a quick regex check into server.js and put in the PR above.

Good news !

Great news @n8henrie.  I know I ran into that same problem as well!  I just took username to actually mean username and set my own (and not in MAC format).  Once I figured out it had to be a valid (but fake) MAC, everything starting taking off.

Perhaps we need an enhancement to check this is a valid MAC address @nfarina ?

@rudders I would support that yes.

Did you guys see my pull request above? I think it should do the trick.

@n8henrie Sorry I missed that one! Looks great. Merging now.

Thanks for that solution @n8henrie !     I could not figure out what happened like so many above and had problems with both my Mac and raspberry Pi.  It looks like homekit somehow locks up a user and the only way is to change the name.  So we have a workaround but do we actually know what happens inside homekit?  The resetting of the homekit configuration should be taking care of this.

have i missed something along the lines here guys, just can't get it to work :(
seeing the Homebridge accessory show in the 'Home' app but when you tap on it I don't get the PIN prompt, just times out after 1m or so.
I've tried:

Modifiying the username/MAC address (to a random 0-9 A-F mix),
Disabling Firewall on Mac
Deleting persistent folder
Resetting HomeKit settings on iOS

would really appreciate any advice at all!

I had an odd situation - my hap_.tcp_. was advertised (twice, two machines) but Home.app was ignoring them. By changing the advertised "name" address, they started appearing. Bonkers.

I have the same problem: it worked perfectly yesterday, and out of nowhere it stopped responding. I rebooted and did not connect any more.
@n8henrie , your solution was to change only the username? But which username? RPi or Homebridge, or iPhone?
Thank you!

Sorry, I haven't looked at this in a while -- I didn't know you could have homebridge-relevant usernames for those other things, I'm pretty sure I just meant the homebridge username.

I seem to have hit this same problem. Changing bridge name, username, and port doesn't seem to get me anywhere. Homebridge is running but my iOS devices can't add it as an accessory even though it's been running well for months prior to this.

I've been struggling on my Mac mini all morning with this like many others. I tried changing the bridge name, username and port ; deleting the .homebrige/persist directory and even  re-installing npm and homebridge, all to no avail.  Something someone mentioned above triggered an answer for me.  My Mac mini has both a wired and wireless connection. As soon as I unplugged the wired connection, everything was fine.
If this helps you, then a thumbs up would be nice.
John Talbot


Clean up and refactor code

remove some duplicate logic

drop main() form internal classes
create common interface for TSS models


Move logic for finding and creating event data to dedicated class

Move logic for finding and creating event data to dedicated class

completed this by creating a Class, EventParser, to manage all the parsing of the event object and return a dictionary to the caller(parsed_event = parser._parse()). However, the function is still marked as complexity5+. Therefore, have to figure out a way to make it less complex. Seems silly to break up the parsing into different functions.
Going to look into some methods for doing this more efficiently. In the meantime, the code is secure, according to CodeQL and Codacity, so closing this and moving on.


I just uploaded and this is happening

RuntimeError: This video was encoded using the VP9 codec. This might be because
the video in question got alot of views. YouBit currently cannot decode this
properly.

That's strange, I never once had a VP9-encoded video during development. I only just added this exception because the 'YouBit Example' video got re-encoded with VP9 because of its views. Can you link me the video URL that caused this?

I added support for VP9 encoded videos in a0834c6.
Upgrade to youbit==0.1.3 to resolve.