lua_package_path ignored by Kong

There is an overwhelming amount of documentation to read through, and read through it I did. All weekend. I thought I understood everything and that I could sit down this morning and write a simple Hello World plugin. I was wrong.
Here was my plan. Write a simple plugin that logs "Hello World" from the new() constructor method of my plugin. That's it. Nothing more complicated.
So I built a docker-compose file which looks like this :
version: "3"
services:
    kong:
        image: kong:latest
        ports:
            - "8000:8000"
            - "8443:8443"
            - "8001:8001"
            - "8444:8444"
        environment:
            - KONG_DATABASE=cassandra
            - KONG_PG_HOST=kongdatabase
            - KONG_PG_USER=kong
            - KONG_PG_PASSWORD=kong
            - KONG_CASSANDRA_CONTACT_POINTS=kongdatabase
            - KONG_PROXY_ACCESS_LOG=/dev/stdout
            - KONG_ADMIN_ACCESS_LOG=/dev/stdout
            - KONG_PROXY_ERROR_LOG=/dev/stderr
            - KONG_ADMIN_ERROR_LOG=/dev/stderr
            - KONG_ADMIN_LISTEN=<IP_ADDRESS>:8001, <IP_ADDRESS>:8444 ssl
        depends_on:
            - kongdatabase
            - nodesoap
        volumes:
            - ./kong:/home/kong:rw
    kongdatabase:
        image: cassandra:3
        ports:
            - "9042:9042"
        healthcheck:
            test: ["CMD-SHELL", "[ $$(nodetool statusgossip) = running ]"]
            interval: 30s
            timeout: 10s
            retries: 5

Note that I just want a super simple setup, where I can keep the code on the host, and have a bind mount.
I created a folder in the host directory thus :
src/plugins/restfulSoap
In this folder I placed 2 files
handler.lua
schema.lua
The contents of these files is irrelevant as the module does not load.
I changed the nginx-kong.conf to include this folder in the lua_package_path variable
lua_package_path       './src/plugins/restfukSoap/?.lua;./?.lua;./?/init.lua;;;;';
Then I set the environment variable thusly :
export KONG_PLUGINS=restfulSoap
The I issued the following commands :
kong prepare
kong reload
In the log file for the docker container I get the following :
2020/10/12 11:59:57 [error] 1#0: init_by_lua error: /usr/local/share/lua/5.1/kong/init.lua:483: error loading plugin schemas: on plugin 'restfulSoap': restfulSoap plugin is enabled but not installed;
module 'kong.plugins.restfulSoap.handler' not found:No LuaRocks module found for kong.plugins.restfulSoap.handler
        no field package.preload['kong.plugins.restfulSoap.handler']
        no file './kong/plugins/restfulSoap/handler.lua'
        no file './kong/plugins/restfulSoap/handler/init.lua'
        no file '/usr/local/openresty/site/lualib/kong/plugins/restfulSoap/handler.ljbc'
        no file '/usr/local/openresty/site/lualib/kong/plugins/restfulSoap/handler/init.ljbc'
        no file '/usr/local/openresty/lualib/kong/plugins/restfulSoap/handler.ljbc'
        no file '/usr/local/openresty/lualib/kong/plugins/restfulSoap/handler/init.ljbc'
        no file '/usr/local/openresty/site/lualib/kong/plugins/restfulSoap/handler.lua'
        no file '/usr/local/openresty/site/lualib/kong/plugins/restfulSoap/handler/init.lua'
        no file '/usr/local/openresty/lualib/kong/plugins/restfulSoap/handler.lua'
        no file '/usr/local/openresty/lualib/kong/plugins/restfulSoap/handler/init.lua'
        no file '/usr/local/openresty/luajit/share/luajit-2.1.0-beta3/kong/plugins/restfulSoap/handler.lua'
        no file '/usr/local/share/lua/5.1/kong/plugins/restfulSoap/handler.lua'
        no file '/usr/local/share/lua/5.1/kong/plugins/restfulSoap/handler/init.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/kong/plugins/restfulSoap/handler.lua'
        no file '/usr/local/openresty/luajit/share/lua/5.1/kong/plugins/restfulSoap/handler/init.lua'
        no file '/home/kong/.luarocks/share/lua/5.1/kong/plugins/restfulSoap/handler.lua'
        no file '/home/kong/.luarocks/share/lua/5.1/kong/plugins/restfulSoap/handler/init.lua'
        no file '/usr/local/openresty/site/lualib/kong/plugins/restfulSoap/handler.so'
        no file '/usr/local/openresty/lualib/kong/plugins/restfulSoap/handler.so'
        no file './kong/plugins/restfulSoap/handler.so'
        no file '/usr/local/lib/lua/5.1/kong/plugins/restfulSoap/handler.so'
        no file '/usr/local/openresty/luajit/lib/lua/5.1/kong/plugins/restfulSoap/handler.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
        no file '/home/kong/.luarocks/lib/lua/5.1/kong/plugins/restfulSoap/handler.so'
        no file '/usr/local/openresty/site/lualib/kong.so'
        no file '/usr/local/openresty/lualib/kong.so'
        no file './kong.so'
        no file '/usr/local/lib/lua/5.1/kong.so'
        no file '/usr/local/openresty/luajit/lib/lua/5.1/kong.so'
        no file '/usr/local/lib/lua/5.1/loadall.so'
        no file '/home/kong/.luarocks/lib/lua/5.1/kong.so'
stack traceback:
        [C]: at 0x7f91343620f0
        [C]: in function 'xpcall'
        /usr/local/share/lua/5.1/kong/tools/utils.lua:616: in function 'load_module_if_exists'
        /usr/local/share/lua/5.1/kong/db/dao/plugins.lua:149: in function 'load_plugin_handler'
        /usr/local/share/lua/5.1/kong/db/dao/plugins.lua:227: in function 'load_plugin'
        /usr/local/share/lua/5.1/kong/db/dao/plugins.lua:275: in function 'load_plugin_schemas'
        /usr/local/share/lua/5.1/kong/init.lua:483: in function 'init'
        init_by_lua:3: in main chunk

Which tells me that it has looked in every place for my pluging, apart from where I configured it to be.
Why does Kong ignore the lua_package_path in the config file?

not sure how you got started. The new() constructor has been deprecated for a while now, instead you should use module level code. Where did you find a reference to it? we might need to fix the docs there...
Have you tried the plugin template and Pongo repositories? Those tools should get you started with custom plugins.

I followed the document link from this repo, then Guides and References > Plugin Development Guide > Implementing Custom logic.
Brings you out here
Then subsection handler.lua specifications
-- Extending the Base Plugin handler is optional, as there is no real
-- concept of interface in Lua, but the Base Plugin handler's methods
-- can be called from your child implementation and will print logs
-- in your `error.log` file (where all logs are printed).
local BasePlugin = require "kong.plugins.base_plugin"

local CustomHandler = BasePlugin:extend()

CustomHandler.VERSION  = "1.0.0"
CustomHandler.PRIORITY = 10

-- Your plugin handler's constructor. If you are extending the
-- Base Plugin handler, it's only role is to instantiate itself
-- with a name. The name is your plugin name as it will be printed in the logs.
function **CustomHandler:new**()
  CustomHandler.super.new(self, "my-custom-plugin")
end

Yes up to date documentation would help :-)
That said, you seem to have missed my question which is why has Kong ignored my setting in the config?
lua_package_path './src/plugins/restfulSoap/?.lua;./?.lua;./?/init.lua;;;;';

You should not edit Kong generated configuration files or files that we ship. Use this: https://docs.konghq.com/2.1.x/configuration/#lua_package_path

I added this to my docker-compose environment variables :
- KONG_LUA_PACKAGE_PATH=/usr/local/custom/?.lua;;
- KONG_PLUGINS=restfulSoap

Then I added a bind mount to /usr/local/custom
- ./plugins:/usr/local/custom/kong/plugins:rw
And now it is attempting to load the plugin, so that addresses this issue, and it can now be marked as closed.
It does leave a couple of loose threads though around the docs, chiefly that they are not providing the latest module definitions for the plugin development section, and that they also instruct the user to edit the config files, which now bungle says is verboten. This now incorrect information can be found under section 3, here https://docs.konghq.com/2.1.x/plugin-development/distribution/
If there is any up to date documentation in the near future please can you comment on this thread so I get an email?

in that case please create an issue here: https://github.com/Kong/docs.konghq.com
I'll close this now.

@suityou01,
Yes, the BasePlugin we deprecated. You can still use it, we have not removed any of it. But we have converted all of our plugins away from it. The docs say that:
Extending the Base Plugin handler is optional

See: https://docs.konghq.com/2.1.x/plugin-development/custom-logic/#handlerlua-specifications
Yes, I agree we should just give an example where there is no BasePlugin.

They also instruct the user to edit the config files,

Yes they instruct you to edit config files. This is our config file:
https://docs.konghq.com/2.1.x/configuration/
You can see the most recent here (by default we search for /etc/kong/kong.conf):
https://raw.githubusercontent.com/Kong/kong/master/kong.conf.default
You can give this config file with: -c option to kong startup:
Usage: kong start [OPTIONS]

Start Kong (Nginx and other configured services) in the configured
prefix directory.

Options:
 -c,--conf        (optional string)   Configuration file.
 --nginx-conf     (optional string)   Custom Nginx configuration template.

Or you can pass the config options as environment variables.
 We should perhaps add a comment at the top of those files that they are auto-generated (so when you do e.g. start, those get rewritten), and do not edit them.
If you really want to edit the generated Nginx templates, we provide you a two ways:

injecting configs in there, e.g. (here I am using env vars, you could also use config file):

# writes "include main.conf;" in main block
KONG_NGINX_MAIN_INCLUDE=main.conf
# writes "include events.conf;" in events block
KONG_NGINX_EVENTS_INCLUDE=events.conf
# writes "include http.conf;" in http block
KONG_NGINX_HTTP_INCLUDE=http.conf
# writes "include proxy.conf;" in http server block
KONG_NGINX_PROXY_INCLUDE=proxy.conf
# writes "include upstream.conf;" in http upstream block
KONG_NGINX_UPSTREAM_INCLUDE=upstream.conf
# writes "include stream.conf;" in stream block
KONG_NGINX_STREAM_INCLUDE=stream.conf
# writes "include stream-proxy.conf;" in stream server block
KONG_NGINX_SPROXY_INCLUDE=stream-proxy.conf
# writes "include stream-upstream.conf;" in stream server block
KONG_NGINX_SUPSTREAM_INCLUDE=stream-upstream.conf
# writes "include status.conf;" in stream server block
KONG_NGINX_STATUS_INCLUDE=status.conf
# writes "include admin.conf;" in stream server block
KONG_NGINX_ADMIN_INCLUDE=admin.conf
# writes "include admin.conf;" in stream server block
KONG_NGINX_ADMIN_INCLUDE=admin.conf

if the above is not enough for you you can use custom nginx template that you can pass with--nginx-conf option.

Our templates that we run when you don't give such a parameter are here:
https://github.com/Kong/kong/tree/master/kong/templates
But as for --nginx-conf you need to give a single file, you need to combine those. In our test suite we have one such example here:
https://github.com/Kong/kong/blob/master/spec/fixtures/custom_nginx.template
I hope this helps. We have notified your request to make docs better, so thank you for suggestions, we'll fix them!


Modify training tutorial to reflect the changes made to the dataloaders

Codecov Report

Merging #191 (ea1e747) into main (9568afa) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #191   +/-   ##
=======================================
  Coverage   88.16%   88.16%
=======================================
  Files          57       57
  Lines        2357     2357
=======================================
  Hits         2078     2078
  Misses        279      279


Sanity checks for the NotBefore / NotAfter dates

In https://github.com/jetstack/cert-manager/pull/3464#discussion_r527566933 @wallrj  wrote:

...But this still leaves (rare) edge cases :

CA might for some reason issue a certificate of 30d1s
administrator sets defaultRenewBefore to 1d which would clash with 1d duration certificate
Developer creates a Certificate with renewBefore 7d and duration 3d, but CA signs certificate for 7d
...or even the case where CA returns a certificate with a NotAfter set in the past
or where NotAfter is set lower than NotBefore

In all these cases there would be weird behaviour and potentially a large number of certificate requests generated.
There should be sanity checks on the NotBefore / NotAfter dates, tests to ensure that we don't ever calculate a negative renewal date  or  a renewal date in the past, and maybe some sort of backoff  to limit the rate of new certificate requests for an individual certificate.

/kind bug
xref: #3012, #3481, #3396

I am also facing similar issues as defined by @wallrj in few deployments. I would request to:

Either we provide some logic to limit the renewal rate of a certificate resource.
Provide some way to lock this value "renewBefore" at the Issuer/ClusterIssuer level.

I would like to suggest this for 1.2 if someone is able to tackle it.

/priority important-soon

/priority important-soon

@wallrj Since we plan on releasing 1.2 tomorrow, would it make sense to bump this issue to 1.3 instead?

/assign

/milestone Next

/milestone v1.3

I'll take a look today!
/assign

Done reviewing!
/unassign

Update: It is almost impossible to know what is sane and what is not because there are always people with specific requirements. We decided to not continue with this work. This issue was initially exacerbated due to the certificate requests not being cleaned up, which has been fixed in #3458.
Future work: the higher-level goal is to avoid cert-manager get into a hot-loop trying to reissue certificates:

Example 1: when the notAfter on the issuer is in the past.
Example 2: you configure your certificate with a duration of 1 year and a renewBefore of 30 days, but what you get back is a certificate valid for 30 days, which means cert-manager will keep retrying it.

/close
/milestone clear


httpdisk-grep chokes on non-gzip files

I was using httpdisk-grep to clean up some failures and ran into this issue.
httpdisk-grep -h "HTTPDISK 403"
httpdisk-grep: not in gzip format (.DS_Store) # << this is unexpected
rm -rf ./**/.DS_Store

httpdisk-grep -h "HTTPDISK 403"
httpdisk-grep: 809: unexpected token at ...

Are you OK with it changing to warn on non-gzip formats as opposed to exiting?

Yeah, I think I warning is fine. I'll catch that exception and print something. BTW, there is another option for grepping that I probably need to document. ripgrep-all with --rga-accurate.

Thanks, Adam. It looks like this should work. I see you bumped the gem version, but maybe didn't get around to publishing the gem, no rush. I have a workaround, but the other updates have been working quite well.

Ha. Will Do. My project is nearing an end, so I'm not sure I have many more.
I have one feature request that I'll document in an issue to consider -- but there is no rush.


DuckDB Comparison - Questions + Bug

Describe the bug
Hi all,
I'm currently writing up a blog post on our recent DuckDB/Arrow integration. I was recommended  to check your project since
(I think) DuckDB and Datafusion are the only systems out there consuming and producing memory Arrow Objects  :-)
I just have a couple of questions and a small bug report.

It seems to me that Datafusion is not consuming streaming arrow objects (i.e., not fully materialized objects), is that correct?
Is datafusion already running in parallel?
Is there a specific batch size that is more attractive to the datafusion engine?

I've managed to install v 0.4  of datafusion/python, from the sources and run the TPC-H Q6. I wanted to check with you, if the numbers I'm getting are expected, or if I did something wrong when experimenting with it (The code I used is under the reproduce)
DataFusion: 0.20
DuckDB: 0.046753292
The bug: One last thing, when trying to generate RecordBatches with different sizes
e.g.:
pq.read_table('lineitemsf1.snappy.parquet').to_batches(1024)

I hit the following exception:

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
/Users/holanda/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-6.2.0/src/util/bit_chunk_iterator.rs:33:9
thread 'tokio-runtime-worker' panicked at 'the offset of the new Buffer cannot exceed the existing length', /Users/holanda/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-6.2.0/src/buffer/immutable.rs:142:9
thread 'tokio-runtime-worker' panicked at 'the offset of the new Buffer cannot exceed the existing length', /Users/holanda/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-6.2.0/src/buffer/immutable.rs:142:9
thread 'tokio-runtime-worker' panicked at 'the offset of the new Buffer cannot exceed the existing length', /Users/holanda/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-6.2.0/src/buffer/immutable.rs:142:9
thread 'tokio-runtime-worker' panicked at 'the offset of the new Buffer cannot exceed the existing length', /Users/holanda/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-6.2.0/src/buffer/immutable.rs:142:9
Traceback (most recent call last):
  File "/Users/holanda/Documents/Projects/tryit.py", line 31, in <module>
    result_df = ctx.sql(query).collect()
Exception: Arrow error: External error: oneshot canceled

Thanks for your time! Always great to learn a bit about the other db engines out there :-)
To Reproduce
from datafusion import ExecutionContext
import pyarrow.parquet as pq
import pyarrow as pa
import time
import duckdb

# wget -q https://github.com/cwida/duckdb-data/releases/download/v1.0/lineitemsf1.snappy.parquet
ctx = ExecutionContext()
conn = duckdb.connect()

query = """SELECT sum(l_extendedprice * l_discount) AS revenue
            FROM
                lineitem
            WHERE
                l_shipdate >= CAST('1994-01-01' AS date)
                AND l_shipdate < CAST('1995-01-01' AS date)
                AND l_discount BETWEEN 0.05
                AND 0.07
                AND l_quantity < 24; """

# for i in range (10,20):
#     batch_size = pow (2,i)
batch_size = '-'
lineitem_fusion = pq.read_table('lineitemsf1.snappy.parquet').to_batches()
lineitem = pa.Table.from_batches(lineitem_fusion)

start_time = time.monotonic()
ctx.register_record_batches("lineitem", [lineitem_fusion])
result_df = ctx.sql(query).collect()
print("DataFusion ("+str(batch_size)+"): "+ str(time.monotonic()-start_time))

start_time = time.monotonic()
result_duck =  conn.execute(query).fetchall()
print("DuckDB ("+str(batch_size)+"): "+ str(time.monotonic()-start_time))

print (result_df[0].column(0))
print (result_duck)

Dandandan summarized it well. Note that there are other systems using Arrow beneath them. E.g.

signavio
IOx
dremio
polars
databend
ask the Arrow dev mailing list for more?

There is also this popular comparison between some of them, https://h2oai.github.io/db-benchmark/ (which is not TPC-H); DuckDB is there :)

There is also a substantial list of "powered by" Arrow systems at https://arrow.apache.org/powered_by/
Something else which may be obvious, but I wanted to make explicit, is that that DataFusion doesn't have its own "native" storage format in the way that DuckDB or other DBMS systems do -- DataFusion is a  query engine that can be used if you have your data in Arrow record batches (or want to load them into memory using register_record_batches).
If you are comparing DuckDB and DataFusion, another comparison might be to start with data in parquet files and compare the timings of:

Time to load the parquet into DuckDB + time to run the query (or time to run the queries in DuckDB if it supports external tables)
The time needed to run the query in DataFusion directly against parquet

Hey guys,  thanks for the quick answers and suggestions :-)
Also, one thing that I really enjoyed about your project is the easy usability, it literally took me a couple of minutes to get a tpc-h query running, coming from database-server-land this always amazes me!
@Dandandan

Hey @pdet
Thanks for opening this issue! I am personally very interested if there are components within the ecosystem that could be shared between engines such as DuckDB and DataFusion and the pyarrow library. I am thinking about query optimization, re-using UDFs, executing certain parts in one engine or the other, etc.

For sure, I think this is one of the most interesting aspects of being able to zero-copy an in-memory data storage format (Exiting Stuff :-)). I'm still considering showcasing an "executing certain parts in one engine or the other", in the blog post, just have to craft a realistic example for that!

For the timings - let me play a bit with your example - I'll come back to it. At first glance, those two things come to mind when looking at the code:

It looks like DuckDB doesn't need to load all columns in memory whereas register_record_batches pre-loads the data (all columns) in memory.
DataFusion can benefit from more parallelism for Parquet reading (See notes on question 2.) by splitting your data into n equal partitions / files.

So, as far as I understand, Arrow has two main types of objects, materialized (e.g., Arrow Tables, RecordBatches ) and streamed ones. (e.g., Record Batch Reader), DuckDB can consume/produce both of these, and quite similar to DataFusion, it automatically parallelizes on the number of batches (If I got that right). In the code-snippet I sent, we are consuming Arrow Tables, exactly to start off from a point where everything is already materialized in memory. From a streaming perspective, yes we wouldn't need to load unnecessary columns/row groups.

Looking forward to the blog post!

It seems to me that DataFusion is not consuming streaming arrow objects (i.e., not fully materialized objects), is that correct?

I am not 100% certain what you mean here. register_record_batches (see PR #981) currently uses MemTable which keeps the data as batches in memory (in a fully materialized list). We don't have a streaming table provider as of now which consumes a stream / iterator of Arrow batches. Also would be interesting to see if we can integrate somehow with an Arrow Table instead.
Yes, my question was again related to record batch readers. You answered it perfectly!

@jorgecarleitao
Thanks for the suggestions, the main point of my post is to use DuckDB as an execution engine for Arrow, so basically I want to compare with other libraries that have bindings to Python/R and can run SQL, while integrating with Arrow Tables/Record Batches/ Record Batch Readers. I'll check if any of those fit these criteria as well! :-)
I'm also familiar with the H2OAI benchmark, I was focusing on TPC-H mainly because of the current blogpost examples, but maybe I should have another look at using H2OAI using as well.
@alamb
Thanks for the list! Will surely have a look.
Just as a clarification, I'm not planning on doing a benchmark comparison with datafusion as engines, but I just want to describe a bit what other options out there can query arrow objects, and a bit of how mature/fast/easy-to-use they are.
Thanks again for all of your answers!


Set up Github Actions

It looks like actions are set up! The tests just need to be fixed (need to add authentication). Will close this issue; CI should be fixed soon.


Cardano sl 0.6

Please don't start MacOS build, this PR should be merged after #1506 (approximately on Saturday).


IOS 10

This doesn't work for me https://github.com/m90/seeThru/issues/39 and the codepen example doesn't play either when i view on browserstack.
I'm doing it the same as on here http://mjbomar.co.nf/index.html, works perfect in all apart from ios and android on browserstack. Any ideas? Known issue? is it cause even though we have auto play the seeThru script is initialising the play?
ok looks like the issue is cause the video is hidden using display: none. I've done a super untested hack on this to force display block using important and then setting the width:0; height:0;

So is that an issue with the library or in your code? Is this still relevant?

I'd say with the library as you are hiding the original video
"video.style.display = 'none'; line 413
This seems to prevent the video from playing with the new ios settings
<video id="video" autoplay loop playsinline muted></video>
I've got round this by doing the following css to keep the video visible to the browser. i've tested this across browserstack ie11+, chrome, ff, ios iphone, android devices. works like a charm but you might have a better solution. Think this would be a great feature to add.
#video {
	display: block !important;
	height: 0;
	position: absolute;
	width: 0;
	z-index: -10;
}

I need to check if the proposed solution generates any unwanted sideeffects in any other setup once I do have access to sth like Browserstack (I don't have actual devices to repro that atm). I'll let you know what I find out.
In the meantime feel free to open a PR against this repo that uses the approach you are describing. Ideally we'd even have something that enables multiple ways of hiding the video (I'd be afraid of the CSS becoming too specific for a library usecase soon). Maybe it'd be a thing to pass in an optional function as an option that will be passed the element and then can do "custom hiding" like in the case you describe.

@adamwright32 I had a quick go at what I just described in this branch: https://github.com/m90/seeThru/tree/cdfa1734743ae946a1b775c4ceaffdecee9728b1
which would basically let you override the setting of display: none by passing in a hash-version of your desired styles if you need it.
Could you check if that works for you and fits your needs?
I'm kind of reluctant of changing the defaults while this is still version 3, but I guess it'd be a good idea to thoroughly research if your proposal is a better fit once this will become a v4.

I'm closing this via v3.2.1 which is enabling the user to pass in custom CSS that is used for hiding the video while being see thru.


Fix typo in README.md

Just saw a little typo in the README as I was scrolling by
No PR too small!

Thanks! btw curious why are there so many ❤ and 🚀 in 10 minutes?

@all-contributor please add pajlada for doc

@all-contributors please add pajlada for doc

Thanks! btw curious why are there so many heart and rocket in 10 minutes?

My group of friends who initially shared this repo with me saw the PR I made and showed their support for the repo :heart:

@pajlada I see :)


Missing Node modules when restoring Yarn cache

I have a setup with Yarn where I have my main dependencies in the root package.json. I also have a submodule in my src folder and I have to install its dependencies as well before building my project.
When the dependencies are installed, my workflow works. However, when they are restored from cache, some of them are missing and the build fails. Specifically, mini-css-extract-plugin is missing. Here are my workflow steps:
- name: Yarn - get cache directory
  id: yarn-cache-path
  run: echo "::set-output name=dir::$(yarn cache dir)"

- name: Yarn - process cache
  id: yarn-cache
  uses: actions/cache@v1
  with:
    path: ${{ steps.yarn-cache-path.outputs.dir }}
    key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }}
    restore-keys: ${{ runner.os }}-yarn-

- name: Yarn - install
  if: steps.yarn-cache.outputs.cache-hit != 'true'
  run: yarn install && cd ./src/js/oblik && yarn install --production=true

- name: Build
  run: yarn build

This is the command that installs the dependencies before they are cached:
yarn install && cd ./src/js/oblik && yarn install --production=true

Basically, the cache is correctly restored, but some modules seem to be missing. Perhaps that --production flag causes some issues? Here's the workflow.log

--production will only install the regular deps and not the dev ones maybe that is the problem?

@zRelux that's what I thought at first as well, but the missing dependency is from the first yarn install that doesn't have the --production flag. The second package doesn't have that dependency at all. I tried without --production and it still failed.

You need to remove if: steps.yarn-cache.outputs.cache-hit != 'true'


#158 Added tests for result callback

First pull request :)

Thanks for the contribution @MattCain
Looks great. I would just make one comment:
Both tests and the "beforeEach" can be simplified by removing the spyOnCommand which tests something unrelated to this test spec.
By removing it:

We'll have simpler and easier to read code for the test.
We prevent the following confusing situation: The code which executes commands gets broken but the tests will report that the match callback functionality failed.

(I know there are some cases in the existing code where I didn't heed my own advice... they need to be improved)

Hi @TalAter
I've just removed the spyOnCommand stuff from the 2 tests for the result callback
I had thought it was necessary as it was also in the tests for the resultMatch and resultNoMatch callbacks, I can also remove it from the those too if that would be helpful?

Merged. Thanks and congrats on your first pull request :heart: :octocat:
Yeah, it looks like those command spies can be removed from those tests as well, thank you.


Dialogue End script only works part of the time (0.7.3)

Describe your problem here.
Some time ago, I was given a dialogue script that lets dialogue play before AND after a song. This is it:
`function onStartCountdown()
if not seenCutscene and isStoryMode then
setProperty("inCutscene", true)
startDialogue("dialogue") --Place file called dialogue in song folder alongside this file
seenCutscene = true
return Function_Stop
end
return Function_Continue
end
function onEndSong()
if not allowEnd and isStoryMode then
setProperty("inCutscene", true)
startDialogue("dialogueEnd")--Place another file called dialogueEND
allowEnd = true
return Function_Stop
end
return Function_Continue;
end`
It works perfectly fine on my custom week 1 and week 2, but only half of it works for the end of my third week. I even tried testing it on a fresh copy of the build but the same thing happens. I have no idea how to fix this or if it CAN be fixed. Any suggestions?
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No response

Turns out the .json was incomplete. My fault.


Task Management Ui

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
Task Management Website Html Css Javascript
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Tech Stack-

HTML
CSS
Javascript

/assign

@Ayush7614 please assign me this issue.


Update Duplicate Prevention Scheme

Prevent duplicate user input by the person's name

There are existing duplicate checking for now, it is better to modify it after basic features in the contact list are done.


Fix announcement overflow

Closes #239
Before:

After:

Pull Request Test Coverage Report for Build 2424

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 96.315%

Totals

Change from base Build 2414:
0.0%

Covered Lines:
575

Relevant Lines:
597

💛  - Coveralls

❤


Investigate warnings about UI updates off the main thread

It looks like this might be something to do with the update lifecycle in SelectableCollectionView.

Definitely something that needs looking at in v1.


Pybits page

Issues:

Make sure you reset to the latest version and rebase your changes on top of it, everytime you contribute. We don't want merge commits.
Commit message is not compliant with the guidelines: 7d176c3

Merging the required commits manually for now as we're running out of time.


rk3588 yolov5 npu占用稍微高一点就会出现乱框

之前使用没有npu超过50%的情况，最近想试试最大的路数时，发现有这个问题

请问下有人遇到过类似情况吗？就是会乱框，没有任何规律，查了模型输出就是这个乱七八糟的结果。

Can you provide the source image.  What YOLO model version are you using?

There must be some other problem as I get the following results for that image.
YOLOv5

YOLOv8

YOLOX

Are you using C++ or Python code?  Is your YOLO model the example one from Rockchip or did you train your own?

There must be some other problem as I get the following results for that image.
YOLOv5
YOLOv8
YOLOX
Are you using C++ or Python code? Is your YOLO model the example one from Rockchip or did you train your own?

Thank you for your response.
Recognition works normally even when identifying a single image or when the frame rate of multiple streams is relatively low.
However, when the number of streams increases significantly, this issue of random bounding boxes occasionally occurs.
I am using C++ code with a custom-trained model.
I will also try testing with the model provided by Rockchip. Thank you for the suggestion.

However, when the number of streams increases significantly, this issue of random bounding boxes occasionally occurs.

If it occurs only when streams and system load increases, then you might have a race condition on a buffer so results from one stream are being rendered on another.

Here are the attempts I made:
1.	Using the yolov5s_relu.onnx model provided in the Yolov5 directory of this project, converting it to RKNN successfully resolved the issue of incorrect bounding boxes.
2.	Then, I trained a model using the Yolov5 branch provided by Rockchip (https://github.com/airockchip/yolov5), exported it to ONNX, and converted it to RKNN. However, the issue of incorrect bounding boxes still persists.
Could you please let me know if the steps in point 2 are correct? Thank you!


Unable to run project

I am getting an issue when trying to load this project by running npm run dev. It seems to go a bit back further than what is deployed, so I think it is an issue with a dep or my node version. Currently trying to figure it out.

Uncaught Error: Cannot find module './src/cat'
    at s (http://localhost:3000/public/bundle.js:1:148)
    at http://localhost:3000/public/bundle.js:1:305
    at http://localhost:3000/public/bundle.js:105426:3
    at Array.forEach (native)
    at Object.<anonymous> (http://localhost:3000/public/bundle.js:105425:28)
    at Object.312../commands.json (http://localhost:3000/public/bundle.js:105555:4)
    at s (http://localhost:3000/public/bundle.js:1:254)
    at http://localhost:3000/public/bundle.js:1:305
    at Object.<anonymous> (http://localhost:3000/public/bundle.js:63090:9)
    at Object.190../log (http://localhost:3000/public/bundle.js:63137:4)
    at s (http://localhost:3000/public/bundle.js:1:254)
    at http://localhost:3000/public/bundle.js:1:305
    at Object.185../lib/constants (http://localhost:3000/public/bundle.js:62898:3)
    at s (http://localhost:3000/public/bundle.js:1:254)
    at http://localhost:3000/public/bundle.js:1:305
    at Object.371.bluebird (http://localhost:3000/public/bundle.js:119416:7)
    at s (http://localhost:3000/public/bundle.js:1:254)
    at http://localhost:3000/public/bundle.js:1:305
    at Object.372../RepoMatrix (http://localhost:3000/public/bundle.js:119857:14)
    at s (http://localhost:3000/public/bundle.js:1:254)
    at e (http://localhost:3000/public/bundle.js:1:425)
    at http://localhost:3000/public/bundle.js:1:443

@harlantwood I narrowed it down. Unfortunately, it looks like your commit https://github.com/ipfs/project-repos/commit/07061a1009475752739eb9dcf1b69fe23db35e84 broke the build, AFAIK - lightsaber 0.6.6 doesn't work as well as 0.5.1. Can you fix lightsaber and update this?

Sorry to hear that!  I'll take a look as soon as I can, but it might be a few days.  Alternatingly, you could just remove lightsaber, and replace itslog with console.log.

SGTM. I've got a workaround at the moment (using an old version), and will merge that as needed.

Cannot replicate on live.


Add scope option to oauth2 refresh function

This adds the option to send scopes in the refreshAccessToken function of the OAuth2 client.
Fixes #18

Can you update the docs as well?

Ofc, to be honest I did this at about 5 in the morning and completely forgot 😅
Will sort now

Docs have been added, think they should cover it

Thanks!


added migrations

When migrations is missing we need create table as migrate --run-syncdb. Otherwise we have problem with some management commands such as flush.

This is a known issue in #121 . #153 is the fix for this that will be landing soon, when I get time off for the Christmas holidays. Until then you can work around by adding migrations in your own project temporarily. Thx.


Not working with later version of React Native (0.47+)

A major change in react native makes the java part break. Remove the @Override on all functions in the MainIntentPackage.java to fix it.

pr2 not working?

@daqiuqu yes it worked! Had to fork it though.


Allow result to be output into a csv

The user should have the option to output the results to a CSV file instead of the default terminal Table

Hello, I would like to work on this issue for hacktoberfest 2024, so could you please assign me this issue?

All yours, @Zingzy. Let me know if you have any questions.

Hello @EndlessTrax I open a PR for this issue, please check 😊


fix: uncomment the 'chrono' feature flag for a function

The line 60 has use chrono::prelude, but the function doesn't have a cfg feature flag.
https://github.com/vityafx/serde-aux/blob/a5b044e1f8e61292cbdd2cdbbdb490794f5ec982/src/field_attributes.rs#L53-L60
Found the following error message in a library that uses serde-aux as a dependency.
serde-aux = { version = "4.1.1", default-features = false }

error[E0433]: failed to resolve: use of undeclared crate or module `chrono`
  --> /home/hari/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-aux-4.1.1/src/field_attributes.rs:60:9
   |
60 |     use chrono::prelude::*;
   |         ^^^^^^ use of undeclared crate or module `chrono`

error[E0433]: failed to resolve: use of undeclared crate or module `chrono`
  --> /home/hari/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-aux-4.1.1/src/field_attributes.rs:56:13
   |
56 | ) -> Result<chrono::DateTime<chrono::Utc>, D::Error>
   |             ^^^^^^ use of undeclared crate or module `chrono`

error[E0433]: failed to resolve: use of undeclared crate or module `chrono`
  --> /home/hari/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-aux-4.1.1/src/field_attributes.rs:56:30
   |
56 | ) -> Result<chrono::DateTime<chrono::Utc>, D::Error>
   |                              ^^^^^^ use of undeclared crate or module `chrono`

error[E0433]: failed to resolve: use of undeclared type `DateTime`
  --> /home/hari/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-aux-4.1.1/src/field_attributes.rs:67:8
   |
67 |     Ok(DateTime::<Utc>::from_utc(
   |        ^^^^^^^^ use of undeclared type `DateTime`

error[E0433]: failed to resolve: use of undeclared type `NaiveDateTime`
  --> /home/hari/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-aux-4.1.1/src/field_attributes.rs:68:9
   |
68 |         NaiveDateTime::from_timestamp_opt(seconds, nanos)
   |         ^^^^^^^^^^^^^ use of undeclared type `NaiveDateTime`

error[E0412]: cannot find type `Utc` in this scope
  --> /home/hari/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-aux-4.1.1/src/field_attributes.rs:67:19
   |
54 | pub fn deserialize_datetime_utc_from_milliseconds<'de, D>(
   |                                                         - help: you might be missing a type parameter: `, Utc`
...
67 |     Ok(DateTime::<Utc>::from_utc(
   |                   ^^^ not found in this scope

error[E0425]: cannot find value `Utc` in this scope
  --> /home/hari/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-aux-4.1.1/src/field_attributes.rs:70:9
   |
70 |         Utc,
   |         ^^^ not found in this scope

Thanks! Was in a rush today, and commented it out for some reason.

@vityafx One more fix left. Give me a second before merging.

@vityafx One more fix left. Give me a second before merging.

Thanks again. I am on a mobile right now, will publish in 20 minutes.

Perhaps, I need to add more checks in CI.

Published https://crates.io/crates/serde-aux/4.1.2
4.1.1 is yanked.
Thanks again!


Abide validation statuses triggered twice.

Provided a simple form as the one from Foundation 5 Docs:
<form data-abide="ajax" id="myform">

        Your name required
            <input type="text" name="users_name" required pattern="[a-zA-Z]+">

        Name is required and must be a string.

    <button type="submit">Submit</button>
</form>

and a simple jQuery piece of code to handle abide validation
$('#myform').on('valid.fndtn.abide', function(e) {

    // Handle the submission of the form
    console.log(e);
});

THAT console.log is always fired twice!
Did any of you encountered the same issue? Any help please?
Thank You.

Same issue here. Changing to 'valid' did the trick. Second issue I've encountered with Abide so far, but good overall.

+1

I believe this issue is cause because of the way jQuery's event namespacing works. Basically, triggering a valid event will fire any listener listening to valid.* events.
Because Foundation is deprecating the valid event and moving to the valid.fndtn.abide event, the current version of the code triggers both during validation, but because of the way jQuery's event works, listener for valid.fndtn.abide will fire for both valid and valid.fndtn.abide event, and since both events fired, the listener ran twice, each for both event.
The obvious solution is to simply trigger the old events during validation, and I actually have prepared a PR with this fix, but I'm not sure what the developer's intentions are, since I noticed they are triggering both versions of the events in other places as well.

Also having this issue.
As mentioned earlier it is due to the way that jQuery event namespacing works.
Simple fix is to check the event namespace.
Like this:
$('form').on('valid.fndtn.abide', function(e) {
    // Fix for valid event firing twice
    if(e.namespace != 'abide.fndtn') {
        return;
    }

    // Handle form...

@patrickgalbraith +1 This solved it. Just remember to take 'e' as input in the function.

@patrickgalbraith cleanest solution I've seen yet, thanks.

@hdra The fix is actually fairly simple instead of triggering the valid event like this:
form.trigger('valid').trigger('valid.fndtn.abide');

Trigger it like this:
form.trigger('valid.$').trigger('valid.fndtn.abide');

This will prevent both from triggering when listening to 'valid.fndtn.abide'.
See this fiddle for example http://jsfiddle.net/z0qfp6yh/

Using just "valid" did the trick. thanks

Using 'valid' worked for me too.

I'm having the same issue as @IshanUpmanyu. Is there a separate ticket for it somewhere? The solutions in this one did not resolve it. I still get multiple events triggering when I submit a form with data-abide="ajax"
I recreated the issue in Codepen: http://codepen.io/TheHanna/pen/XJyxXJ
Pop open the console and click submit on the form--you'll see the number of events logged increase by one each time you click.

@TheHanna this is because you are binding the "fndt" event each time that you press the submit button. You don't need the onclick event, this work is made already by abide.

Thanks @diegoviola1, that was it.

@patrickgalbraith Thanks for that fix. Thought I was going nuts.

I found a simpler solution that works even if you have a dynamic form. This is a combination of Foundation's JavaScript utilities.
This will work as follows:

Validate the fields (assuming your fields has "required" tags and "error" classes)
If valid, prevent from submitting same content for 1000 milliseconds
(Optional) Clear the fields (add your function to clear all fields)
Re-validate

$(document).on('valid.fndtn.abide','#myForm', Foundation.utils.debounce(function(e) {// Handles the Submission

    //ajax function here
      console.log('I am not fired twice');
    //clear all fields for re-validation after successful submission

}, 1000, true));

I did this and it fixed the issue:
$(entry[0]).on("submit valid.fndtn.abide", function (event) {
// Handle the submission of the form
            event.preventDefault();
        });

Using jQuery's .one() method resolves this issue for me.
$('#form').one('valid.fndtn.abide', function() {

// Handle valid form...

});

The  .one() method is identical to  .on(), except that the handler is unbound after its first invocation.


Nightly runs to detect access artefacts drift

The solution should provide a config option during deployment where if the parameter is set to true, then the solution would:

run a nightly job that would detect if there's a difference between what the solution holds versus what the AWS SSO configuration reflects with regards to permission sets and account assignments.
If delta is detected, then the solution should do one of the two remediation actions depending on the option set by the user in configuration:

Notify an email address chosen in the config for every drift detected
Auto remediate to ensure that AWS SSO matches the solution repo with regards to permission sets and account assignments.

@vpegg to update on current status of the PR


Kani does not detect UB when generating an invalid slice reference from an invalid slice pointer

I tried this code:
//! Converting a raw pointer to a reference should trigger UB if the metadata is not valid.
#![feature(set_ptr_value)]

// Generate invalid fat pointer by incrementing the pointer address without adjusting the metadata.
#[kani::proof]
fn check_invalid_read() {
    let data = "hello";
    let ptr = data as *const str;
    // This should trigger UB since the metadata does not get adjusted.
    let val = unsafe { &*ptr.byte_add(1) };
    assert_eq!(val.len(), data.len());
}

// Generate invalid fat pointer by combining the metadata.
#[kani::proof]
fn check_with_metadata() {
    let short = [0u32; 2];
    let long = [0u32; 10];
    let ptr = &short as *const [u32];
    // This should trigger UB since the slice is not valid for the new length.
    let fake_long = unsafe { &*ptr.with_metadata_of(&long) };
    assert_eq!(fake_long.len(), long.len());
}

using the following command line invocation:
kani invalid_fat_reference.rs

with Kani version: 0.54.0-dev
I expected to see this happen: Both harnesses should fail due to UB. Note that both cases fail in MIRI.
Instead, this happened: Verification succeeds

I believe this is related to #2975, but we probably forgot to check for fat pointers.


🐛 Wildcard and name patterns emit broken code.

Closed by https://github.com/ren-lang/compiler/commit/ac3a8d8e60020c27542f310b28c5304654b8651b.


Question about interaction layer

I am trying to find how interaction layer is implemented.
But I had no success.. could you help me with interaction layer?
I searched for it in model > InteractionModels
and there seems to be no code for model architecture.
I read your paper and found out that the interaction layer composes of transformer decoder.
How can I implement this?

Thanks!

I cannot find this code in the repo. Are your results for DR-BERT(as mentioned in the paper) extracted from this model or some other?


Error occurs when launching the lio.launch.py.

Hello. Bother you again.
I build the li_slam_ros2 successfully but there is an error when I use the command "ros2 launch scanmatcher lio.launch.py". The detailed info is shown below. Can you help me solve it? Thank you!
[INFO] [launch]: All log files can be found below /home/cakgod/.ros/log/2020-11-09-03-25-29-689650-cakgod-ThinkCentre-M910x-N000-75592 [INFO] [launch]: Default logging verbosity is set to INFO Task exception was never retrieved future: <Task finished name='Task-2' coro=<LaunchService._process_one_event() done, defined at /opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py:271> exception=SubstitutionFailure("executable 'imu_preintegration' not found on the libexec directory '/home/cakgod/ros2_ws/install/scanmatcher/lib/scanmatcher' ")> Traceback (most recent call last):   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py", line 273, in _process_one_event     await self.__process_event(next_event)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py", line 293, in __process_event     visit_all_entities_and_collect_futures(entity, self.__context))   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures     futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures     futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 45, in visit_all_entities_and_collect_futures     futures_to_return += visit_all_entities_and_collect_futures(sub_entity, context)   [Previous line repeated 1 more time]   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 38, in visit_all_entities_and_collect_futures     sub_entities = entity.visit(context)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/action.py", line 108, in visit     return self.execute(context)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch_ros/actions/node.py", line 442, in execute     ret = super().execute(context)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/actions/execute_process.py", line 775, in execute     self.__expand_substitutions(context)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/actions/execute_process.py", line 635, in __expand_substitutions     cmd = [perform_substitutions(context, x) for x in self.__cmd]   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/actions/execute_process.py", line 635, in <listcomp>     cmd = [perform_substitutions(context, x) for x in self.__cmd]   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/perform_substitutions_impl.py", line 26, in perform_substitutions     return ''.join([context.perform_substitution(sub) for sub in subs])   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/perform_substitutions_impl.py", line 26, in <listcomp>     return ''.join([context.perform_substitution(sub) for sub in subs])   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_context.py", line 184, in perform_substitution     return substitution.perform(self)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch_ros/substitutions/executable_in_package.py", line 84, in perform     raise SubstitutionFailure( launch.substitutions.substitution_failure.SubstitutionFailure: executable 'imu_preintegration' not found on the libexec directory '/home/cakgod/ros2_ws/install/scanmatcher/lib/scanmatcher'  Task exception was never retrieved future: <Task finished name='Task-12' coro=<LaunchService._process_one_event() done, defined at /opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py:271> exception=RuntimeError('Signal event received before subprocess transport available.')> Traceback (most recent call last):   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py", line 273, in _process_one_event     await self.__process_event(next_event)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/launch_service.py", line 293, in __process_event     visit_all_entities_and_collect_futures(entity, self.__context))   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/utilities/visit_all_entities_and_collect_futures_impl.py", line 38, in visit_all_entities_and_collect_futures     sub_entities = entity.visit(context)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/action.py", line 108, in visit     return self.execute(context)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/actions/opaque_function.py", line 75, in execute     return self.__function(context, *self.__args, **self.__kwargs)   File "/opt/ros/foxy/lib/python3.8/site-packages/launch/actions/execute_process.py", line 421, in __on_signal_process_event     raise RuntimeError('Signal event received before subprocess transport available.') RuntimeError: Signal event received before subprocess transport available. [INFO] [scanmatcher_node-1]: process started with pid [75594] [INFO] [static_transform_publisher-2]: process started with pid [75596] [INFO] [scanmatcher_node-1]: sending signal 'SIGINT' to process[scanmatcher_node-1] [ERROR] [scanmatcher_node-1]: process has died [pid 75594, exit code -2, cmd '/home/cakgod/ros2_ws/install/scanmatcher/lib/scanmatcher/scanmatcher_node --ros-args -r /input_cloud:=/cloud_deskewed --params-file /home/cakgod/ros2_ws/install/scanmatcher/share/scanmatcher/param/lio.yaml']. [static_transform_publisher-2] [INFO] [1604910329.767936466] [static_transform_publisher_I3XGArIQqPVe3muV]: Spinning until killed publishing transform from 'base_link' to 'velodyne' [ERROR] [static_transform_publisher-2]: process[static_transform_publisher-2] failed to terminate '5' seconds after receiving 'SIGINT', escalating to 'SIGTERM' [INFO] [static_transform_publisher-2]: sending signal 'SIGTERM' to process[static_transform_publisher-2] [ERROR] [static_transform_publisher-2]: process has died [pid 75596, exit code -15, cmd '/opt/ros/foxy/lib/tf2_ros/static_transform_publisher 0 0 0 0 0 0 1 base_link velodyne --ros-args'].

Btw, I use the ubuntu 20.04 and ros2 foxy.

Maybe you don't have gtsam installed or linked to li_slam_ros2.
Make sure you can build this package by commenting out 「if(GTSAM_FOUND)」 and 「endif()」 below.
https://github.com/rsasaki0109/li_slam_ros2/blob/master/scanmatcher/CMakeLists.txt#L102-L105
https://github.com/rsasaki0109/li_slam_ros2/blob/master/scanmatcher/CMakeLists.txt#L134-L179

Thanks. It works after reinstalling the GTSAM.
There is another question. Did this framework support other lidar datasets well? For example, the 16/32/64 layers dataset. Have you test it on such datasets?
Looking forward to hearing from you.

I think this package would work with other datasets as well, but I haven't tried it. I think it works, at least with the data described in the LIO-SAM README.md.

Hi! Thank you for your great work!
I was working on looking for a good ros2-based slam frameworks and testing
them. Some ros1-based slam frameworks including hdl_graph_slam, lego_loam,
and lio-sam had been test. You can find the test report in the project
slam_test https://github.com/CAKGOD/slam_test. Some ros2-based slam
frameworks including lidarslam_ros2 and li_slam_ros2 were also test.
(Without any offense, I found that it was harder in parameter adjustments
about your framework. I believe it is because of my insufficient ability.)
I need to select a ros2-based slam framework from them to contribute to
ros2 community. Of course, there is a way for me to do this. Now there are
two ways to complete it. One is porting the lio-sam to ros2 and the other
is contributing the li_slam_ros2 (if it is agreed). But I found some
matters in the porting process of lio-sam such as the unporting of pcl_ros.
So we can contribute the li_slam_ros2 if it has better slam ability and you
agree with it of course.
So I want to know your opinion on this.
Ryohei Sasaki<EMAIL_ADDRESS>于2020年11月13日周五 上午4:21写道：

I think this package would work with other datasets as well, but I haven't
tried it. I think it works, at least with the data described in the LIO-SAM
README.md.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rsasaki0109/li_slam_ros2/issues/2#issuecomment-726648884,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AH3ICCWH7ESPX2KK4G2CT2LSPT275ANCNFSM4TPA3HPA
.

Maybe the essential problem is not parameter tuning.
The reason it doesn't work with the KITTI you're testing is because I developed this package for a mobile robot that uses a 16-line lidar.
The accuracy of my package is equal to or better than other SLAM packages (I think NDT / GICP is considerably more accurate than LOAM). The problem, however, is that the processing is heavy.
The package I am using is not very suitable for cars. For car scan matching, it is very important to estimate the initial value from the previous position and the previous two positions, but this feature is not implemented in this package because it is not important for mobile robots. Without this feature, the NDT calculation time is even longer and real-time scan matching in the vehicle is more difficult
So I find it difficult to process KITTI data in real time.
Maybe slowing down the Rosbag would make it a little better.

Thank you for your reply.


ssh conection down when "docker-compose up --build"

Creating network "docker_backend" with driver "bridge"
Creating network "docker_frontend" with driver "bridge"

No resources for basic support.


Update rxjs to version 7.0.0

This pull request was created using the JSFIX tool (https://jsfix.live) by Coana.tech (https://coana.tech).
It bumps rxjs to version 7.0.0.
Sign up your project here to receive notifications about other packages updates JSFIX can help with.

 Pull request details
 🚧 - Manual review items (please check before merge)
Breaking changes not automatically fixable by JSFIX.

General breaking changes (not related to specific API usages in your code).

onUnhandledError: Errors that occur during setup of an observable subscription, after the subscription has emitted an error, or completed will now throw in their own call stack. Before it would call console.warn. This is potentially breaking in edge cases for node applications, which may be configured to terminate for unhandled exceptions. In the unlikely event this affects you, you can configure the behavior to console.warn in the new configuration setting like so: import { config } from 'rxjs'; config.onUnhandledError = (err) => console.warn(err);
TS: RxJS requires TS 4.2
RxJS Error types Tests that are written with naive expectations against errors may fail now that errors have a proper stack property. In some testing frameworks, a deep equality check on two error instances will check the values in stack, which could be different.

The following breaking changes are unlikely to affect your code. Below each bullet is listed the dependency usages detected by JSFIX that may be affected by the breaking change.

map: thisArg will now default to undefined. The previous default of MapSubscriber never made any sense. This will only affect code that calls map with a function and references this like so: source.pipe(map(function () { console.log(this); })). There wasn't anything useful about doing this, so the breakage is expected to be very minimal. If anything we're no longer leaking an implementation detail.

bin/concurrently.spec.ts#L43-L43
src/flow-control/input-handler.ts#L42-L42
src/flow-control/kill-on-signal.ts#L28-L31
src/flow-control/kill-others.ts#L36-L36

take and will now throw runtime error for arguments that are negative or NaN, this includes non-TS calls like take().

src/completion-listener.ts#L86-L86
src/flow-control/log-timings.ts#L79-L79
src/flow-control/restart-process.ts#L36-L36

throwError: In an extreme corner case for usage, throwError is no longer able to emit a function as an error directly. If you need to push a function as an error, you will have to use the factory function to return the function like so: throwError(() => functionToEmit), in other words throwError(() => () => console.log('called later')).

src/completion-listener.ts#L84-L84

Additional details
Breaking changes where JSFIX patched all occurrences.

toPromise: toPromise return type now returns T | undefined in TypeScript, which is correct, but may break builds.

src/completion-listener.ts#L78-L88

Calling .next on a Subject without passing a value is no longer allowed

src/flow-control/log-timings.spec.ts#L102-L102

Breaking changes where JSFIX found that there were no occurrences.

Subscription: add no longer returns an unnecessary Subscription reference. This was done to prevent confusion caused by a legacy behavior. You can now add and remove functions and Subscriptions as teardowns to and from a Subscription using add and remove directly. Before this, remove only accepted subscriptions.
Subscriber: new Subscriber no longer takes 0-3 arguments. To create a Subscriber with 0-3 arguments, use Subscriber.create. However, please note that there is little to no reason that you should be creating Subscriber references directly, and Subscriber.create and new Subscriber are both deprecated.
Leaked implementation detail _unsubscribeAndRecycle of Subscriber has been removed. Just use new Subscription objects
ReplaySubject no longer schedules emissions when a scheduler is provided. If you need that behavior, please compose in observeOn using pipe, for example: new ReplaySubject(2, 3000).pipe(observeOn(asap))
rxjs-compat: rxjs/Rx is no longer a valid import site.
The static sortActions method on VirtualTimeScheduler is no longer publicly exposed by our TS types.
Notification.createNext(undefined) will no longer return the exact same reference everytime.
race: race() will no longer subscribe to subsequent observables if a provided source synchronously errors or completes. This means side effects that might have occurred during subscription in those rare cases will no longer occur.
unsubscribe no longer available via the this context of observer functions. To reenable, set config.useDeprecatedNextContext = true on the rxjs config found at import { config } from 'rxjs';. Note that enabling this will result in a performance penalty for all consumer subscriptions.
defer no longer allows factories to return void or undefined. All factories passed to defer must return a proper ObservableInput, such as Observable, Promise, et al. To get the same behavior as you may have relied on previously, return EMPTY or return of() from the factory.
mergeScan: mergeScan will no longer emit its inner state again upon completion.
single operator will now throw for scenarios where values coming in are either not present, or do not match the provided predicate. Error types thrown have also been updated, please check documentation for changes.
An undocumented behavior where passing a negative count argument to repeat would result in an observable that repeats forever.
Removed an undocumented behavior where passing a negative count argument to retry would result in an observable that repeats forever.
skipLast: skipLast will no longer error when passed a negative number, rather it will simply return the source, as though 0 was passed.
takeLast now has runtime assertions that throw TypeErrors for invalid arguments. Calling takeLast without arguments or with an argument that is NaN will throw a TypeError
zip: zip operators will no longer iterate provided iterables "as needed", instead the iterables will be treated as push-streams just like they would be everywhere else in RxJS. This means that passing an endless iterable will result in the thread locking up, as it will endlessly try to read from that iterable. This puts us in-line with all other Rx implementations. To work around this, it is probably best to use map or some combination of map and zip. For example, zip(source$, iterator) could be source$.pipe(map(value => [value, iterator.next().value])).
zip: Zipping a single array will now have a different result. This is an extreme corner-case, because it is very unlikely that anyone would want to zip an array with nothing at all. The workaround would be to wrap the array in another array zip([[1,2,3]]). But again, that's pretty weird.
ajax body serialization will now use default XHR behavior in all cases. If the body is a Blob, ArrayBuffer, any array buffer view (like a byte sequence, e.g. Uint8Array, etc), FormData, URLSearchParams, string, or ReadableStream, default handling is use. If the body is otherwise typeof "object", then it will be converted to JSON via JSON.stringify, and the Content-Type header will be set to application/json;charset=utf-8. All other types will emit an error.
The Content-Type header passed to ajax configuration no longer has any effect on the serialization behavior of the AJAX request.
ajax: In an extreme corner-case... If an error occurs, the responseType is "json", we're in IE, and the responseType is not valid JSON, the ajax observable will no longer emit a syntax error, rather it will emit a full AjaxError with more details.
ajax: Ajax implementation drops support for IE10 and lower. This puts us in-line with other implementations and helps clean up code in this area
some internal modules are no longer exported (not described in changelog)
pairs: pairs will no longer function in IE without a polyfill for Object.entries. pairs itself is also deprecated in favor of users just using from(Object.entries(obj)).
count: No longer passes source observable as a third argument to the predicate.
That feature was rarely used, and of limited value.
The workaround is to simply close over the source inside of the function if you need to access it in there.

If you would like to provide feedback to the JSFIX developers, then please leave a comment on this pull request.

Coverage remained the same at 97.903% when pulling 2eeee42f6fc3f012b0a867fcced8edb7b458d876 on jsfix-updater:jsfix-updated-rxjs-to-v7.0.0-47 into 74412badf0eb5402dc83fdca74843eeb305304fd on open-cli-tools:main.


Create API for Saving and Loading Model Snapshot

I want to try out different training mechanisms using save-restore-replay. Could you tell me if it's possible to make API to support such use-cases?

Today you have a few approaches to save the model and continue work with that

Save the model as TF graph and weights in txt format (example) The model could be used for inference only after loading as InferenceModel.
Save the model as JSON config and weights in txt format (example). The model could be fine-tuned, training could be continued. This approach works well in both cases: you trained your model in Keras or KotlinDL firstly


Updates fail because it runs using are run using yarn instead of npm to perform the update

My @types/node package was out of date. So I ran npm-check -u and it threw this error:
$ yarn add<EMAIL_ADDRESS>--dev --color=always
Command failed with ENOENT: yarn add<EMAIL_ADDRESS>--dev --color=always
spawn yarn ENOENT
For more detail, add `--debug` to the command

It seems to be using yarn instead of npm.
How can I make it update using npm instead of yarn?

Try NPM_CHECK_INSTALLER=npm npm-check -u.

Why is yarn the default instead of npm for a tool called npm-check ? This is confusing and annoying as it used to work fine without NPM_CHECK_INSTALLER...
Is there at least a way to permanently make npm the default ?

@lionel-bijaoui yeah it's a weird behaviour

This package uses another package to detect the package manager to use: preferred-pm
Package preferred-pm is having a problem with identifying the package manager, maybe because you are using NPM Workspace?
https://github.com/zkochan/packages/issues/180

Package preferred-pm is having a problem with identifying the package manager, maybe because you are using NPM Workspace?

I have an npm monorepo, and therefore I am using NPM Workspaces. So that could be it!


hfss and hfls interchanged

A typo in the parameter table ifspar.json (and perhaps also the f90 namelist?) causes the latent and sensible heat fluxes to be switched.

rev.475d3883900301069f82e0fc070e731f1f0a7216


🐛 [firebase_dynamic_links] getInitialLink() always returns null when terminated app launched on iOS

Bug report
Describe the bug
When the application is launched from a dynamic link on iOS the getInitialLink() method always returns null. Shortly after, the onLink callback is executed.
Steps to reproduce
Steps to reproduce the behavior:

Set up & configure app following instructions: https://pub.dev/packages/firebase_dynamic_links
Launch iOS app from dynamic link.
Notice that the getInitialLink() returns null but onLink is called shortly after.

Expected behavior
getInitialLink() returns deeplink when app is launched from a terminated state on iOS.
Additional context
Adding a one second delay before calling getInitialLink() fixes the issue.

Flutter doctor
Run flutter doctor and paste the output below:
Click To Expand
PASTE OUTPUT INSIDE HERE

Flutter dependencies
Run flutter pub deps -- --style=compact and paste the output below:
Click To Expand
PASTE OUTPUT INSIDE HERE

Hi @EddieWilkie91
This sounds like a duplicate of https://github.com/FirebaseExtended/flutterfire/issues/6913 so closing it as such.
Let me know if you disagree.
Thank you

Hi @EddieWilkie91 This sounds like a duplicate of #6913 so closing it as such. Let me know if you disagree. Thank you

Yeah, that's no probs. I thought it would be slightly different because my onLink method for iOS gets called after I recieve null from getInitialLink(). But happy to leave it and continue watching the other issue. Cheers.


[Bug]: page.screenshot (silently) times out when trying to take a screenshot of an alert

Bug expectation
I expected that screenshot would never time out (it doesn't even have an option to set the timeout time, as most other Puppeteer functions do).
But, if it were to timeout, I'd expect it to do so in a reasonable amount of time, with an error message explaining why it timed out.
And really I'd expect screenshot to just take a screenshot of the alert ... or take a "screenshot" of a black screen and the text "alerts can't be screenshoted" ... but if that's not possible, then failing in a fixed amount of time with an error message would be nice.
And if nothing else, please at least add a line to https://pptr.dev/api/puppeteer.page.screenshot documenting this.
Bug behavior

[ ] Flaky
[ ] PDF

Minimal, reproducible example
import puppeteer from 'puppeteer';

(async () => {

    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto(SOME_PAGE);
    // This button should trigger a window.alert
    // Also note that "click" *does* have a timeout option
    // (unlike screenshot)
    await page.click('button', { timeout: 2000 });
    await page.screenshot({ path: 'screenshot.png' })
    // wait for your test runner to time out

})();

Error string
None: no error is ever thrown (that's a big part of the bug)
Puppeteer configuration
No response
Puppeteer version
19.7.2
Node version
v19.7.0
Package manager
npm
Package manager version
9.5.0
Operating system
Linux

This is a general issue with CDP unfortunately, if the renderer is paused and automation tries to get information from it, the browser never resolves. Timeouts unfortunately has not been consistently part of the Puppeteer API. In the best case, this should be fixed on the both sides, the browser and Puppeteer.

Does it work if you intercept the dialog? E.g.,
page.on('dialog', async dialog => {
    console.log(dialog.message());
    await dialog.dismiss();
});

How to break the cloudflare in nodejs and multiple page loading

Timeouts unfortunately has not been consistently part of the Puppeteer API.

Don't all of the waitFor* methods take a timeout option?  Since there's no waitForScreenshotToBePossible method, wouldn't it make sense to add one ... or better yet, just give screenshot a timeout option?
This seems easy to add code-wise; here's my ghetto version:
const screenshotWithTimeout = async (page, options) => {
  let wasSuccessful = false;
  setTimeout(() => {
    if (!wasSuccessful) throw new Error('screenshot timed out');
  }, options.timeout);
  delete options.timeout;
  const shot = await page.screenshot(options);
  wasSuccessful = true;
  return shot;
};

I'd be happy to submit a PR (of a less ghetto version) if desired.

I don't think there's anything special about the screenshot method here, so the question/feature request seems to boil down to "why don't all Puppeteer methods have a timeout, including non-waitForX methods?" and "can more/all Puppeteer methods offer a timeout?"

It's definitely possible to add timeout parameters too all API but I think it's more ergonomic if we make the CDP client respect the default timeout (https://pptr.dev/api/puppeteer.page.setdefaulttimeout/) for CDP calls.  That should resolve issues when the timing out is caused by the browser and not an internal bug in Puppeteer.

Great news, thanks OrKoN!


File structure and CRUD

Hi,
I was wondering how this file structure would work with CRUD requests.
I created pages for users like so:

And I also created a module called users with all the funtionality. Although I feel it is the wrong approch to this structure. Is it possible you could add some code to show how to do this correctly?
Also what if I try to "show" a user and the API returns a 404? I find that there is no way to handle it apart from redirect to the page not found page, which isn't what I want to do. Instead I'd like to keep the URL the same and display the page not found.
Thanks for any help in advance.
Jamie

I use the same structure as in your screenshot. If you want to catch errors from your api, and still keep the same page just use a try catch for the axios request and show the appropriator message in that page or maybe an alert.


feat:Added automation for checking commit style

This will help beginners to follow the commit style which Keptn is following.
closes #94
Signed-off-by: Tarang Verma<EMAIL_ADDRESS>

It can be seen here..

This is a GitHub Action that ensures your PR title matches the Conventional Commits spec.

https://github.com/amannn/action-semantic-pull-request#action-semantic-pull-request
@tarangverma this PR adds support for checking the PR title which is a good thing to have but it won't fix https://github.com/keptn-sandbox/datadog-service/issues/94 . You can create a separate Github issue for checking PR title and we can say this PR fixes that issue.

i think am done with the changes..

And merged!


Widget API metatables

The methods described in https://wowpedia.fandom.com/wiki/Widget_API are not present on the objects themselves, but instead on metatables that are shared among all objects with the same type. That is, the following invariant holds:
foo:GetObjectType() == bar:GetObjectType() iff getmetatable(foo) == getmetatable(bar)

This probably also means that we need to change the handling of virtuals and mixins in XML interpretation. Today, virtuals create new uiobject types, with any attached mixins considered the same as Widget API methods from core types. This may or may not be correct, and will require testing in a real client.

A few other observations:

API methods are all in a single flat metatable---no chaining.
The invariant seems to imply that the entire set of methods is available at construction time. Any inherited onload handlers would see the whole object including methods from subtypes. This probably just means I'm overthinking things and am too tied to the current implementation where mixing-in and constructors happen side-by-side and in inheritance order.


fix(plugins/shared-schemas): remove 400 response message suppression

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
598

Relevant Lines:
598

💛  - Coveralls


How to implement SSL ?

Hello!
First, thanks for this really nice package :)
I’m interested in implementing SSL for secure API communication and wanted to ask if you have any guidance on setting this up.
thanks,
Paulin

Hi Paulin! Thanks for the message.
What you want is a reverse proxy that handles the incoming traffic and forces the TLS handshake. Typically people use either nginx or caddy the latter being easier to use in my experience. The reverse proxy server is used to capture traffic on port 443 (which is required for TLS).
The reverse proxy is responsible for handling the https. Once that is successful you would direct it to the port of the main valve app which is 3000 by default. Then valve will be responsible for spinning up and down new instances as needed.
Here is a pretty bad diagram to explain the workflow:

@andrewheiss has a really great example of using caddy to serve a plumber API https://github.com/andrewheiss/docker-plumber-caddy/tree/main
You would need to modify the plumber-thing folder to use valve. Here is a vignette on Using Valve with Docker. If you do get this done, a PR with a vignette and example would be most welcome!

It actually looks like it might be fairly easy to set up TLS natively in Valve if people come with their own pem files https://github.com/tokio-rs/axum/blob/main/examples/tls-rustls/src/main.rs

hi @JosiahParry,
Thank you for all the details! I’ll look into them.


EventProcessingModule should lazily initialize processors

The eager initialization of event processors can cause circular dependencies when one of the event handlers relies on the configuration. This is espectially the case when using an external dependency injection mechanism to wire some beans.
Event Processors are initialized at the latest when the configuration starts, or earlier when the processors are retrieved from the configuration.

Feeling: call to initializeProcessors() before calling ensureInitialized()? Now you're initializing after ensuring everything is initialized... Maybe naming issue?


Requesting option for switching to Legacy DartPad

I'm experiencing performance issues with Sketch_pad. It feels slow and unresponsive when I use it.

Thanks for opening an issue @s4nk37. Would you mind opening specific issues for the issues you're seeing with the new frontend, including system and browser details? That way we can address the performance issues if possible. In terms of the old UI, refer to https://github.com/dart-lang/dart-pad/issues/2892. Thanks again!


Look into creating task with video and csv files

Are multiple files supported?

https://labelstud.io/guide/ts+video.html
https://github.com/AI-Sensus/Experiments/issues/15

For task creation one can create a JSON containing the url of the imu data (.csv) and the video (.mp4). For more information, see: https://labelstud.io/guide/ts+video.html

There has been chosen to use the csv+mp4 syncs trick proposed above. This created issue: #114


Error: out of memory

Checklist

[X] I have read COMMON_ISSUES.md
[X] I have searched existing issues

Description
Have tried running node index.js several times but I keep getting the same error.
An entire folder of assets is not included in the final image mix.
Steps to reproduce
Replace images run node.js
Expected result
Several of your images are not included in the final result
Actual result
Images without some layers
Environment
Winddows

@juansebsol

An entire folder of assets is not included in the final image mix.

Do you mean a layer isn't included? Is the layer still present in build/json/?

Error: out of memory

Is this related?


s3a error

in the example I have attached the following problem appears, which seems to be related to the management of shuffle in spark in the s3 context.
Did I confirm that the problem occurs or is it a configuration problem of mine?
ShuffleExample.scala.zip

this is the main error in cloudwatch
Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find any valid directory for output-

18:48:23
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext$DirSelector.getPathForWrite(LocalDirAllocator.java:541)

18:48:23
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:627)

18:48:23
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.createTmpFileForWrite(LocalDirAllocator.java:640)

18:48:23
at org.apache.hadoop.fs.LocalDirAllocator.createTmpFileForWrite(LocalDirAllocator.java:221)

18:48:23
at org.apache.hadoop.fs.s3a.S3AOutputStream.(S3AOutputStream.java:91)

18:48:23
at org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:736)

18:48:23
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:914)

18:48:23
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:895)

18:48:23
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:792)

18:48:23
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:781)

18:48:23
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.writePartitionedFileToS3(BypassMergeSortShuffleWriter.java:269)

18:48:23
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.writePartitionedFile(BypassMergeSortShuffleWriter.java:223)

18:48:23
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:200)

18:48:23
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)

18:48:23
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)

18:48:23
at org.apache.spark.scheduler.Task.run(Task.scala:99)

18:48:23
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)

18:48:23
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

18:48:23
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6

if I provide the configuration information I entered but followed the documentation

Hey @webroboteu
I remember facing this issue during its development. I wanted to get back to this issue and fix it but if you fast upload of s3a it should work fine. Can you try setting this flag spark.hadoop.fs.s3a.fast.upload  true if its not already set? Also there have been lot of changes in the lambda environment from then to now like the private VPC and things like that which you might have seen in the other issues.

I had already tried these parameters without success. Now out of desperation I was thinking of bypassing the hadoop interface and managing the stream directly. Is your email on linkedin the one you posted on your profile? I would like to have you on my network to discuss the project

ll try again and let you know

if you want to recompile it you suggest to use your hadoop version 2.6.0-qds-0.4.13 but not the reference to your repository. Can you suggest something about version 2.8 for example?

Right. But you can just compile with the existing open source 2.6.0 hadoop version and just copy the hadoop-aws jar later to your binary that should work as well. This is a comment I added in another issue Compiling #2
Another easier workaround is to remove the pom.xml additions basically reverting the commit "Fix pom.xml to have the other Qubole repository location having 2.6.0... (2ca6c68)"

Build your package using this command - ./dev/make-distribution.sh --name spark-lambda-2.1.0 --tgz -Phive -Phadoop-2.7 -DskipTests

And finally add the below jars to classpath before starting spark-shell

1. wget http://central.maven.org/maven2/com/amazonaws/aws-java-sdk/1.7.4/aws-java-sdk-1.7.4.jar
2. wget http://central.maven.org/maven2/org/apache/hadoop/hadoop-aws/2.7.3/hadoop-aws-2.7.3.jar
Refer here - https://markobigdata.com/2017/04/23/manipulating-files-from-s3-with-apache-spark/

recompiling as you say I have the following error:
Exception in thread "dag-scheduler-event-loop" java.lang.NoSuchMethodError: com.amazonaws.http.AmazonHttpClient.disableStrictHostnameVerification()

I have a repository of a docker image: https://github.com/webroboteu/sparklambdadriver
I'm using hadoop version 2.7 and its dependencies.

With hadoop 2.9, referring to bundle 1.11.199 with these docker lines there is progress but I still have to confirm that it works on lambda context
RUN wget http://central.maven.org/maven2/com/amazonaws/aws-java-sdk-bundle/1.11.199/aws-java-sdk-bundle-1.11.199.jar
RUN wget http://central.maven.org/maven2/org/apache/hadoop/hadoop-aws/2.9.0/hadoop-aws-2.9.0.jar
RUN rm /$SPARK_HOME/jars/aws*.jar
RUN mv aws-java-sdk-bundle-1.11.199.jar /$SPARK_HOME/jars
RUN mv hadoop-aws-2.9.0.jar /$SPARK_HOME/jars

with local execution now i have this problem: java.lang.NullPointerException
at org.apache.spark.util.Utils$.localFileToS3(Utils.scala:2517)
at org.apache.spark.shuffle.S3ShuffleBlockResolver.writeIndexFileAndCommit(S3ShuffleBlockResolver.scala:177)
at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:158)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
at org.apache.spark.scheduler.Task.run(Task.scala:99)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:282)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
I update you

I'm in the right direction since I can now recompile it correctly. For some strange reason try to load the data from the same executorId<PHONE_NUMBER>
java.io.FileNotFoundException: No such file or directory: s3: //webroboteuquboleshuffle/tmp/executor-driver-4775351731/30/shuffle_0_0_0.index


ng-controller with tabs

Hi
I'm using tabset and I want each tab to have his on controller.
The problem is that tab is creating his own scope and if will sepcify the controller on the tab's DOM i will receive an error that multiple directives asking for new/isolated scope.
One fix is to set the controller in a div inside the tab, but in that way I can't pass a function from the controller to the "select" callback of the tab.
Is there any workaround for my problem?
Thanks

In the future, please do not use the Issues queue for support and usage questions.  Questions of this nature will get a better response on Stack Overflow, please see the Need Help section here.
If I understand your question correctly, you want to use the tab's select callback to call a controller function in your tab's controller?  What are you trying to accomplish that makes you feel you need to do this?  At first glance, it sounds like you're trying to call an initialize function.

You are better off creating a custom directive with directive controllers for your tabs - it leads to cleaner HTML.
We do not want to recommend using ng-controller for this purpose, as it is an antipattern.

Also have the same problem, I have separate controller for content of each my tab, I want call some controller function from DOM using "select" attribute.
wesleycho, I do not understand, how can I use my custom directive with your tab/tabset controls (directives)? Is it possible to combine, to use your component in my case or I need to write own tab logic by myself? Please give some example if that possible.

@SergeyKulgan, it's not clear what exactly you want to do from a UX/UI perspective.  Can you be more clear?  Thanks.

@icfantv, thanks for quick response. Please point correct way, how can I use your uib-tabset/uib-tab directives to reach this:

   ...

            ...
            here I also using some scope properties from parentCtrl
            ...

   ...

Where initializeTabA() - function, defined in $scope of tabACtrl controller, it downloads some content, should be executed on tab selecting, thus tabACtrl should be accessible from "select" attribute of tab.

Per @wesleycho's suggestion above, don't do this.  You need to create a directive and put that in your uib-tab element.  We've seen issues surrounding errors due to the replace: true property on our directives such that angular will complain about not being able to find the appropriate controller for the uib-tab or uib-tabset directives.
You can then use the observer pattern to implement inter-directive communication (don't use events).

Thanks for suggestion, but, what I think about it:
It is so complicated...I need just execute method on tab select, this is the most wanted function and should be out from the box, It should be so easy...I'm .NET developer, have also simple sites on Angular. It was never so hard for me to make such simple things in .net... Why I can not make this tab as part of my tab class (controller) and somehow (in my class) set handler for exposed onSelect event? I do not need directive, not going reuse it, it is not like control. And, as I understood, to implement inter-directive communication I need to modify source code of that uib-tab directives (to add methods for event handlers registration and call that handlers in needed place). I'm consumer/user and should not modify code of base components to add such functionality, again in terms of OOP, it should be encapsulated (I, as user not interested about internal details and implementation and do not want to dig in it), all that I need: interface/api to use it and have most used functions out from the box, without dance with tambourine. I dislike that angular directives approaches, they far from ideal and contradicts with OOP, I'm sure that somehow it can be done much more better.
BTW, I completely read and understood http://blog.carbonfive.com/2015/03/25/communication-between-collaborating-directives-in-angular/

@SergeyKulgan, IMO, far too often, we as engineers think tend to think about things at a macro level.  E.g., I want to do x, how do I do it?  Sometimes, a better solution is to think about it from a UX and end-user perspective.  E.g., I would like to do x when the user does y.  Perhaps if you start there I can be of further assistance.

I come from an OOP background (Java) as well and the thing you need to realize is that Javascript is not an OO language so you can't think of things in terms of OOP.  Yes, you can kinda/sorta do it, but it's not the same thing.  When you did .ASP you had full control over everything, which is why, in general, it wasn't hard to do stuff like executing a callback when the user clicks something because there's no real sense of scope.  And if you used jQuery, you exposed two of the core problems that angular attempts to solve:  1) being able to look at a component and know exactly what happens when you interact with it (because the callback bindings are right there), and 2) automatic two-way data binding so you don't have to do stuff like:  $('#myInputField').val() to get some input data or $('#myStaticDiv').text('snoopy').
But angular also gives you the framework to break stuff up into smaller functional pieces - something you'd have to do all on your own in jQuery - and part of that is following the rules of how to properly use those functional pieces.  I admit it's a bit esoteric at first and there's not a lot of great resources on how to properly design/architect an enterprise application - rather the books/tutorials tend to focus on how to use one small piece of the API.

Yes, I agree, 2-way data biding and declarative approach, when you looking on directive element in html, it like an object, name is intuitive and allows you to understand what it do, attributes- like properties of that object and implementation is hidden/encapsulated in directive js. It is cool no doubt. But angular very strict, interaction between directives, controllers are restricted, we have easy access parent from child but not vise-versa or between siblings, but in real they should often communicate. Hard to model in accuracy real objects and their relations, often I need some workaround, angular puts you in a strict matrix, if you want to implement some simple and popular - it very easy with angular, but you loosing flexibility and it becomes even more difficult when you want to make some difficult custom functionality. Anyway, thanks for answer, I will learn angular more.

I used following to solve that problem:

     <uib-tab heading="Tab A" select="initializeTabA()">
         ....
     <uib-tab>

or can use directive instead:

     <tabA-directive></tabA-directive>

where template of tabA-directive contains:
<uib-tab heading="Tab A" select="initializeTabA()">
      ....
<uib-tab>

and directive controller is tabACtrl, which scope contains initializeTabA() function, it will be executed on tab select.

Be careful with that latter approach - you can break the Bootstrap CSS which is, unfortunately, tightly coupled to DOM structure (i.e. they use direct child selectors > rather than the descendent selector).

All of these suggestions are great, but it looks like you're all taking the simplest approach possible in an attempt to close this issue without actually addressing the problem at hand.
Your suggestion @wesleycho to create a directive to avoid the ng-controller approach is fantastic. I would like to extend on that and say not only is it good for an increase on performance but to encapsulate each tabs properties and controller methods opens up component reusability and separation of concerns which we should all be looking for in our applications.
Now @icfantv from the last suggestion @SergeyKulgan has taken the advice from @wesleycho and created these custom directives for each tab except this won't work as bootstraps css rules force a particular DOM structure which is impossible to achieve with any form of directive approach.
Now if you're following along you will see the crux of this support ticket.
You've altered a core component of this library without testing the range of ways it was used in previous versions, and then instead of coming up with an answer to the problem you've just suggested a bunch of 'best practises' this guy should adhere to that are literally impossible to use due to what i've mentioned above.
@SergeyKulgan What you want to do in order to solve this problem is create a new set of styling rules from the default bootstrap.css, anything that references the following:
.nav-tabs > li {}

.nav > li {}

you want to create in your css file the same rules but with a middleman, this middleman can be any given class/selector that you apply to the element in which your directive is set to:
.nav-tabs > .foo > li {}
.nav > .foo > li {}

For example:
<tabset>
  <tab-one class="foo"></tab-one>
  <tab-two class="foo"></tab-two>
</tabset>

This is the cleanest approach to resolve this problem.
Feel free to re-open this ticket and close it now that a real solution has been given.
Thanks


Feature: typescript support, fixes

First of all - Thank you for your project!
What this PR includes:

[feature]: adding typescript support
[fix]: keep title width if showTitle is false
[fix]: DEFAULT_TIMES_WIDTH_PERCENTAGE in order to make it to be a percentage instead of points

@korifey91 wow thanks! great PR!
I have a couple of questions:

Why the index.d.ts is in .gitignore? (I have little experience in TS 😅)
Can you please revert the version change in package.json? @hoangnm can change that when publishing to npm
Can we rename the interface Event to interface EventItem? I think just "Event" is too broad. Other suggestions are appreciated as well, e.g. interface WeekViewEvent (cc @hoangnm)

Oh, actually I don't remember why I added it to eslintignore 😅 I'll remove that.
Sure thing, I'll make all amendments you mentioned!

@pdpino if you don't mind I close this PR and open a new one in order to make all changes we discussed not in master branch since I use it in my project.


share your dataset

your project are perfect, I very like, can you share your dataset? by the way, how tu run the project? I hope you give me suggestion, thanks

your project are perfect, I very like, can you share your dataset? by the way, how tu run the project? I hope you give me suggestion, thanks

ok ，first of all， we need some jdk to make it happen or py“files”


TextBox tab char not consistent when using mono-spaced font

Left: Avalonia, Right: WPF
I have found that tab chars are not consistent with WPF text rendering using same font mono-spaced (Consolas, 12).

@wieslawsoltes related to this, how do we set the width a tab char renders to?

Tested with 0.6.0:

The tabs are not rendered correctly on netcoreapp2.0 on Windows 10.

I noticed that the Direct2D backend supports rendering tabs correctly now :)

Just tried with 0.10 and seems TextBlock and TextBox are unable to display tabs. @Gillibald should they be able to?

Tabs depending on some column layout that needs to be defined. Without any special processing, they are just some whitespace.

WPF has a default value for incremental tab stops: https://docs.microsoft.com/en-us/dotnet/api/system.windows.media.textformatting.textparagraphproperties.defaultincrementaltab?view=net-5.0#System_Windows_Media_TextFormatting_TextParagraphProperties_DefaultIncrementalTab that we could support as well. Transforming a tab stop to a fixed with of spaces is kind of trivial to implement.

For reference: https://github.com/dotnet/wpf/blob/master/src/Microsoft.DotNet.Wpf/src/PresentationCore/System/Windows/Media/textformatting/TextParagraphProperties.cs#L121
https://github.com/dotnet/wpf/blob/master/src/Microsoft.DotNet.Wpf/src/PresentationCore/MS/internal/TextFormatting/SimpleTextLine.cs#L1597

How WPF treats tab stops in general: https://github.com/dotnet/wpf/blob/master/src/Microsoft.DotNet.Wpf/src/PresentationCore/MS/internal/TextFormatting/TextMetrics.cs#L98

Is there any workaround for this currently? I'm working on a project and tabs are displayed as a little box in a question mark as someone else mentioned.

Fixed in master


Fix up Key-press for API call

Current up keypress throws this error
INFO:pyvizio.cli:Emulating pressing of 'up' key
ERROR:pyvizio:Failed to execute command: unexpected status BLOCKED: Operation blocked
INFO:pyvizio.cli:ERROR

I have checked for the correct code from heathbar/vizio-smart-cast API call and this one is correct

thanks for the submission! have you tested this change on your device?

Yes I have, it's now working flawlessly.


Update commons-io version used in test to the one used in camel-parent (2.11.0)

On CI, we're failing MavenTest on commons-io:commons-io:2.8.0 - I'm not exactly sure why this is failing but, I'm going to try to patch it by upgrading the version of commons-io to the version being used in camel-parent.

This seems to fix the MavenTest error in CI - we should do this.