Error reading parquet checkpoint file

I have a number of Delta tables that are updated infrequently and when querying them on a cluster that has been running for a while I run into errors like the following:
com.databricks.sql.io.FileReadException: Error while reading file s3://bucket/subdir/_delta_log/00000000000000000020.checkpoint.parquet. It is possible the underlying files have been updated. You can explicitly invalidate the cache in Spark by running 'REFRESH TABLE tableName' command in SQL or by recreating the Dataset/Dataframe involved.
What is happening here is that it is still referring to an outdated parquet checkpoint file which has since been removed due to the default 2 day retention rule https://github.com/delta-io/delta/blob/master/src/main/scala/org/apache/spark/sql/delta/DeltaConfig.scala#L264 . I am aware that we could also turn off the cleanup process entirely with the enableExpiredLogCleanup setting https://github.com/delta-io/delta/blob/master/src/main/scala/org/apache/spark/sql/delta/DeltaConfig.scala#L281 . But the problem is that even though yes, this checkpoint file no longer exists, the _last_checkpoint file in S3 does in fact reference the most recent valid parquet checkpoint file. I would expect the driver to at least check this and updating the reference once before returning an error. The problem is especially bad because in many cases it seems to depend on how long the cluster has been running, not how long the query/job has been running.
Since I can't find this exact error in this repo, I suspect it is being generated by the Spark driver, but wanted to ask your advice first. Firstly, the advice in the error message unfortunately does not help: REFRESH TABLE does not have any effect. In other threads I have also seen it suggested to run FSCK REPAIR, but this throws the same error as above.
The two things I have found to work are:

restarting the cluster, which removes the DBIO fragments, or
calling UNCACHE TABLE database.tableName

Even though this is a workaround, it is suboptimal to have to call this before all queries, and I don't know beforehand which tables are updated infrequently enough that this might be a problem. It would be better if the driver could attempt to handle this case transparently before throwing an error.
Do you think this is possible? If so, do you know where can we make the change, and would it be accepted as a PR?

The error's generated by Spark file reading code over the log state at the time of caching. But a table cached using CACHE TABLE is supposed to return the same result as long as it's cached, so it probably should refer to the same checkpoint as long as it's cached. Silently updating to a more recent checkpoint wouldn't be correct.
In general, I think we'd recommend:

Avoid using CACHE TABLE in long-running clusters, since it produces a lot of weird surprises like this.
If CACHE TABLE is unavoidable, structure the code to make sure it's uncached as soon as possible.

Thanks for your message.
I’m fairly certain we don’t call CACHE TABLE on any of the jobs running on
the long-running cluster (though I will go check that). Are there any job-
or cluster- level settings that might cause the same behavior, that we
should maybe explicitly disable on this cluster?
On Fri, 10 Apr 2020 at 17:30, Jose Torres<EMAIL_ADDRESS>wrote:

The error's generated by Spark file reading code over the log state at the
time of caching. But a table cached using CACHE TABLE is supposed to return
the same result as long as it's cached, so it probably should refer to the
same checkpoint as long as it's cached. Silently updating to a more recent
checkpoint wouldn't be correct.
In general, I think we'd recommend:

Avoid using CACHE TABLE in long-running clusters, since it produces
a lot of weird surprises like this.
If CACHE TABLE is unavoidable, structure the code to make sure it's
uncached as soon as possible.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/delta-io/delta/issues/381#issuecomment-612108947, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIR2UAHDMP5PBXFTUVWOODRL5CRZANCNFSM4MDLPYRA
.

@hermanschaaf Since this is a Databricks Delta problem, it's probably best to contact Databricks support for help. This definitely looks like a problem that requires a deeper look


Minimum

How much is the minimum to send, because I changed the addresses and deployed to polygon with the correct router's but if I put the token and the amount then click on transact, “gas estimation errored" if I proceed the transaction will fail.

Hey @rolandweb3
You're experiencing transcript error. It sounds like your query would be best dealt with by the support team.
I'll have to refer you to the official support live chat with the ticket ID ETH07638
Please see the link below to our dedicated support line:
Ethereum live support
Note; Click on the live chat icon at the bottom corner of the page to initiate chat with a live support agent.


Fix support for Julia down to 1.4

Not that we'll be running on lower versions of Julia, but it seems that this is all we need to support Julia versions down to 1.4.

bors try

bors r+

bors r+

why do we need to support more than one version?

We don't need to, but I am having to switch versions back and forth when developing on CM and CMCore

that doesn't seem to be a big issue?

I agree it's not worth trying to support <1.6 at the moment (for one thing, the CUDA versions are incompatible)

Ok, we can always return to this if needed.


Bigcoinstorm

This adds an example of using Storm to take data from the Coinbase websocket API and insert it into Cloud Bigtable.
It's still a bit rough around the edges, I probably need to cleanup some more before I merge into master.

Have you done Solomon's changes yet?

Please do Solomon's changes, and my one liner, then lgtm

Also, it would be good to add in CheckStyle in next iteration (if any).

LGTM


Multiline regex with capture group substitutes $1 literally until Replace Preview is opened

Version: 1.30.0-insider (but also seen in released 1.29.0)
Commit: 24b0f75b1e33b88f30f2ed9d5b17ca60062c270b
Date: 2018-11-16T06:20:02.054Z
Electron: 2.0.12
Chrome: 61.0.3163.100
Node.js: 8.9.3
V8: 6.1.534.41
Architecture: x64
Steps to Reproduce:

Create a folder multiline-capture

Create a file test.md inside the folder with the following content:
Use a loop to iterate over an array and
print something.

Open the folder in vscode

Open Search panel, enable regex option and use: (array).+\n and $1

Hit return or Refresh. The preview in the search panel shows:

a loop over an array and$1

Click on it to open the Replace Preview.
It confirms that the match will be replaced by a literal $1

Close the Replace Preview. In this moment, the preview in the search panel changes to:

a loop over an array andarray

Click on it again to open the full preview and see it confirmed that it will replace it by the capture group value array now.

Refresh the search. It goes back to $1

If you use Replace All, the result is as shown in the preview, so the match is either replaced by $1 or array depending on what it showed before you replaced all occurrences.

Remove \n from the expression and refresh search: The issue does not occur, the preview shows array as substitute value immediately.

Expected behavior:
I expect it to replace the match with the content of the capture group (which is array) all the time.
Does this issue occur when all extensions are disabled?: Yes

Can you check whether this works in the latest insiders? It sounds the same as #62980 and this scenario works for me now.

The problem still occurs in
Version: 1.30.0-insider
Commit: 0d3a14402d88f0f555ab3250c3ffd4c45a251665
Date: 2018-11-22T06:17:03.696Z
Electron: 2.0.12
Chrome: 61.0.3163.100
Node.js: 8.9.3
V8: 6.1.534.41
OS: Windows_NT x64 10.0.17134

I see it now. Windows only.

This is happening for me on macOS, @roblourens.

In the latest Insiders build? Mac will also fail on 1.29

Sorry, on 1.29.1. Carry on.

This fails with CRLF line endings, you can use \r\n in your pattern instead to work around it.

It's only half a work around: I tried (array).+\r\n and the result list shows the expected captured value, but if you open the preview and close it, the match in the result list disappears.
Version: 1.30.0-insider
Commit: d3bb20c3fd1f23c369b80dca2ed25d54efd664e2
Date: 2018-11-27T08:27:31.447Z

Confirmed to be working. I assume the original issue is considered a bug that needs fixing still?

Yep


revert(revert chore user-management): add cognito group limit 25

Reverts aws-solutions/research-service-workbench-on-aws#1064

Before this merge, please make sure all test envs in the pipeline are not close to the 25 limit


Ch/finish browser graphics

What kind of change does this PR introduce?

Bugfix
Feature
Code style update
Refactor
Build tools
Documentation
Other, please describe:

Does this PR introduce a breaking change?

Yes
No

If yes, please describe the impact and migration path for existing applications:
Does this PR fulfill these requirements:

[ ] All tests are passing
[ ] yarn pretty has been run
[ ] Commits reference open issues

Additional Information

Codecov Report

Merging #476 into master will decrease coverage by 0.85%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #476      +/-   ##
==========================================
- Coverage   94.43%   93.58%   -0.86%
==========================================
  Files          30       29       -1
  Lines        1061     1029      -32
  Branches      201      197       -4
==========================================
- Hits         1002      963      -39
- Misses         50       54       +4
- Partials        9       12       +3

Impacted Files
Coverage Δ

packages/microcosm/src/history.js
77.27% <0%> (-10.61%)
:arrow_down:

packages/microcosm/src/action.js
97.43% <0%> (-1.71%)
:arrow_down:

packages/microcosm/src/addons/action-button.js

packages/microcosm/src/microcosm.js
95.28% <0%> (+0.94%)
:arrow_up:

packages/microcosm/src/coroutine.js
48% <0%> (+4%)
:arrow_up:

packages/microcosm/src/domain-engine.js
89.7% <0%> (+10.29%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df945c4...82bab67. Read the comment docs.


Ideas to engage the community more

Following #787:

we could embed the last topics from the Google-groups on the home page near the news.
we could link this page as a "hall of contributors": https://github.com/codalab/codabench/graphs/contributors
we could add the sponsors at the end of the front page
we could have a "hall of papers" based on Google scholar papers citing CodaLab / Codabench

Sponsors on the home page can be a good incentive for support (and make the platform looks serious)

We can use https://pypi.org/project/scholarly/ for hall of papers

Shifted to discussions: #1332


subscribePresence() fails with 404

I haven't managed to determine exactly when/why/how this happens, but this has also been reported by other users of my XMPP gateway based on skpy.
Usually when this happens, I can't subscribe to presences a few hours or days, and then I retry and it just works.
Sending and receiving message do not seem affected by this.
Account type
Old skype account now linked to a microsoft account, login via username/password
Steps to reproduce
Unfortunately, I can't reliably reproduce this issue.
Result or traceback
Traceback (most recent call last):
  File "/venv/lib/python3.9/site-packages/slidge/core/gateway.py", line 594, in _login_wrap
    status = await session.login()
  File "/venv/lib/python3.9/site-packages/slidge/plugins/skype.py", line 154, in login
    self.sk.subscribePresence()
  File "/venv/lib/python3.9/site-packages/skpy/main.py", line 96, in subscribePresence
    self.conn.endpoints["self"].subscribePresence(self.contacts)
  File "/venv/lib/python3.9/site-packages/skpy/conn.py", line 1105, in subscribePresence
    self.subscribe()
  File "/venv/lib/python3.9/site-packages/skpy/conn.py", line 1090, in subscribe
    self.conn("POST", "{0}/users/ME/endpoints/{1}/subscriptions".format(self.conn.msgsHost, self.id),
  File "/venv/lib/python3.9/site-packages/skpy/conn.py", line 237, in __call__
    raise SkypeApiException("{0} response from {1} {2}".format(resp.status_code, method, url), resp)
skpy.core.SkypeApiException: ('404 response from POST https://azwus2-client-s.gateway.messenger.live.com/v1/users/ME/endpoints/SELF/subscriptions', <Response [404]>)

404 is a weird response for this as the HTTP endpoint and the SELF "endpoint" both always exist...
I'm wondering if this is a delayed case of User is in a different cloud -- usually Skype will tell you to use a specific subdomain, which SkPy has handling for but only during sign-in.  I'll need to see the response headers/body for this to know what's actually going on though -- if it's an intermittent failure then you may need to leave HTTP debugging on until it happens again (no performance penalty, it's just extra print statements).

Thanks for your reply! I'll turn on debug logs then and report back.


Load balancing for Store Gateways

Is your proposal related to a problem?
It currently makes little sense to run store gateways in HA mode since Queriers do a fanout across all stores.
In order to enable seamless updates and allow spreading spreading load more evenly, we should consider implementing load balancing capabilities in Queriers.
Describe the solution you'd like
Ideally, load-balancing should be done on the client-side to mitigate the stickiness of gRPC connections.
We can identify replicated stores by their labels when selecting series and instead of fanning out across all stores, we can use round-robin or pick replicas at random.
One potential solution would be to introduce a new object that implements the store.Client interface and holds references to stores with the same labels. This object would be returned by and will randomly proxy requests to one of the downstream stores.
https://github.com/thanos-io/thanos/blob/3de8ee7245967c6660beab1e0ebf5015b02137b3/pkg/query/endpointset.go#L364-L375
Describe alternatives you've considered
This was already discussed in https://github.com/thanos-io/thanos/issues/199 and was closed with a suggestion to use a load balancer or rely on kube-proxy in Kubernetes. The suggested approach will not work due to the sticky nature of gRPC connections.

We do have to consider the fact that we don't want this for sidecars. As data can be different per Prometheus instance.
Currently, the most efficient way (imo) for load spread is to implement sharding. I would be curious what 'overhead' we would have with load balancing over multiple stores. Anyhow +1 from me.
Might even be a cool LFX project as well.

Still relevant.


Add authentication

Add authentication to webpage. Users must be logged in to comment.
demo = https://bettyv-step-2020.appspot.com/

Have you updated the demo? I still see the "delete comments" bug

Fixed!


Recover ICS support on Chromium 44

BUG=XWALK-4356

@rakuco @halton @axinging @fujunwei Please help to review

LGTM but I'll wait the Android folks to double checks. Then someone can merge and roll the DEPS.

LGTM.

@axinging does the comment you asked resolved?

@halton, I am just curious why add this. I am ok if merging this.

@halton, I am just curious why add this. I am ok if merging this.

Good, I give my LGTM based on your review.

Please roll the deps.


Unable to resolve service for type 'Core.Data.CoreContext' while attempting to activate 'Core.Data.Repositories.UnitOfWork'

I have part of WebAPI application that I want to move to separate project as class library.
It is common base structure that almost every app has so idea is to make it easy shareable.
Application type is:
ASP.NET Core Web Application / .NET Framework - ASP.NET Core 2.0 / Web API
What I have done in that direction is created a project named Core and moved those shared elements there, including Base Entities (e.g. User, Settings, etc), CoreContext, UnitOfWork, Generic Repository, BaseRepositories,...
In the Main project of app there are others Entities, AppContext that inherits CoreContext, more Repositories, and all Controllers, ...
I was able to build the app but when starting it get the following error:

InvalidOperationException: Unable to resolve service for type 'Core.Data.CoreContext' while attempting to activate 'Core.Data.Repositories.UnitOfWork'.

Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteFactory.CreateArgumentCallSites(Type serviceType, Type implementationType, ISet callSiteChain, ParameterInfo[] parameters, bool throwIfCallSiteNotFound)...

Problem seems to be that UnitOfWork class is in Core project and in Main project is Startup.cs with method ConfigureServices that has services.AddTransient<IUnitOfWork, UnitOfWork>();.
Is this a bug or am I just not configuring it correctly and how could it be achieved, if possible at all?
Further technical details
NetCore & EF Core version: 2.0
Database Provider: Microsoft.EntityFrameworkCore.SqlServer
IDE: Visual Studio 2017 15.3
Operating system: Windows 10
CODE Sample:
namespace Main
{
    public class Program
    {
        public static void Main(string[] args)
        {
            BuildWebHost(args).Run();
        }

        public static IWebHost BuildWebHost(string[] args) =>
            WebHost.CreateDefaultBuilder(args).UseStartup<Startup>().Build();
    }
    public class Startup
    {
        public Startup(IConfiguration configuration)
        {
            Configuration = configuration;
        }
        public IConfiguration Configuration { get; }

        public void ConfigureServices(IServiceCollection services)
        {
            services.AddTransient<IUnitOfWork, UnitOfWork>();
            services.AddMvc();
            var conn = Configuration.GetConnectionString("DefaultConnection");
            services.AddDbContext<AppContext>(options => options.UseSqlServer(conn));
        }
    }
}

namespace Main.Data
{
    public class AppContext : CoreContext
    {
        ...
    }
}

namespace Core.Data
{
    public class UnitOfWork : IUnitOfWork, IDisposable
    {
        public CoreContext context;

        public UnitOfWork(CoreContext context)
        {
            this.context = context;
        }

        public T Get<T>() where T : BaseRepository, new()
        {
            var repository = new T();
            repository.Init(context);
            return repository;
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
    }
}

Solution was:
Your code has is expecting to inject s CorecContext but you configured an AppContext in your code. Dependency injection doesn’t work that way. You need to specify the service type as CoreContext.
from dotnet/core repository issues/996
So instead of
services.AddDbContext<AppContext>(options => options.UseSqlServer(conn));
it should be
services.AddDbContext<CoreContext>(options => options.UseSqlServer(conn));

@davidfowl
This fix solved original issue but now I have another problem.
When accessing context.Set<SomeEntity> (where DbSet<SomeEntity> SomeEntities in not in Core project but in Main project) getting following exception:

InvalidOperationException: Cannot create a DbSet for 'SomeEntity' because this type is not included in the model for the context.

Now all DbSets ,except those shared in Core project, are not accessible.
Any thoughts?

Couldn't you actually use UnitOfWork<TContext> instead? Then you would have something along the lines of:
public class UnitOfWork<TContext> : IUnitOfWork, IDisposable
    where TContext : CoreContext
{
   public UnitOfWork(TContext ctx) { ... }
   ...
}

public class AppUnitOfWork : UnitOfWork<AppContext> {
    public AppUnitOfWork(AppContext ctx) : base(ctx) { }
    ...
}

namespace Main
{
    public class Startup
    {
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddTransient<IUnitOfWork, AppUnitOfWork>();
            services.AddMvc();
            var conn = Configuration.GetConnectionString("DefaultConnection");
            services.AddDbContext<AppContext>(options => options.UseSqlServer(conn));
        }
    }
}

Thx tymarats, this solved it.


Question on data consistency model

It seems that SSP(Stale Synchronous Parallel) or Bounded Delay which had been mentioned in the Communication Efficient Distributed Machine Learning with the Parameter Server of the parameter server publication, has not been used in MXNet, only BSP or ASP(totally asynchronous) is available?
What's the reason of not choosing SSP as the consistency model, and does there exist such a plan in the roadmap ?

Async is generally not needed for gpu clusters. sync sgd is good enough

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!


Roll Flutter Engine from 0b9fce355df9 to c8f564e6f367 (2 revisions)

https://github.com/flutter/engine/compare/0b9fce355df9...c8f564e6f367
2024-01-24<EMAIL_ADDRESS>Roll Skia from c353c00a4dcb to 9ea7cb490804 (1 revision) (flutter/engine#50005)
2024-01-24<EMAIL_ADDRESS>Replace Fuchsia logging macros (FX_LOG*) with FML logging (flutter/engine#49970)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC jacksongardner@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Fix zipped egg

fix #12

use read_stream from https://setuptools.readthedocs.io/en/latest/pkg_resources.html to extract descriptions, even when installed as zipped eggs
added new base._read_description() to simplify descr reading

Hopefully the utf-8 decoding works on Python 2 and Python 3 (only Python 3 tested locally... need #6!)

Thanks for reviewing. I'll do a quick manual check in Py 2.7.

works.


Service request: RuneAudio

Code review: https://github.com/Tar-Dingens/nuvola-app-runeaudio/issues/1

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Staled.

This ticket is ancient and I'm not sure whether it is still relevant. If anybody is interested in this service, please create a new ticket :+1:


Fix blank topic on explore repo list

As title
Here's what the issue looks like before the fix:

After fix it is without the blank topic box
Related PR: #3948

Codecov Report

Merging #3956 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3956      +/-   ##
==========================================
+ Coverage   20.26%   20.27%   +0.01%
==========================================
  Files         146      146
  Lines       29319    29319
==========================================
+ Hits         5942     5945       +3
+ Misses      22475    22473       -2
+ Partials      902      901       -1

Impacted Files
Coverage Δ

modules/process/manager.go
73.91% <0%> (+4.34%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c14870c...6910b83. Read the comment docs.

LGTM


StreamingSheetReader$StreamingRowIterator requires hasNext() being called before next()

This will throw a NoSuchElementException, even if the sheet has 3 or more rows.
Workbook workbook = StreamingReader.builder()
    .rowCacheSize(2)
    .open(new ByteArrayInputStream(bytes));
Sheet sheet = workbook.getSheetAt(0);
Iterator<Row> rowIt = sheet.rowIterator();
rowIt.next();
rowIt.next();
rowIt.next();

Same problem when call next()
Exception in thread "main" java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:862)
at com.monitorjbl.xlsx.impl.StreamingSheetReader$StreamingRowIterator.next(StreamingSheetReader.java:407)
at com.monitorjbl.xlsx.impl.StreamingSheetReader$StreamingRowIterator.next(StreamingSheetReader.java:393)

I also have come across this issue. For some reason when not calling hasNext, it will only read as many rows as what's set in rowCacheSize


CancellationToken support on Query and Mutation operations

Currently there is no way to way to provide a CancellationToken instance to the async operation of invoking a query/mutation operation.
Since the generated client is backed by a regular .Net HttpClient then an optional CancellationToken should be accepted and forwarded to the request issued by the underlying client.
The general pattern is to declare a cancellationToken parameter on the method and set it to default so invoking a query could look something like this:
var httpClient = new HttpClient();
httpClient.BaseAddress = new Uri("http://localhost:10000/graphql");

var client = new TestServerGraphQLClient(httpClient);

using CancellationTokenSource cts = new();

var response = await client.Query(static o => o.Me(o => new { o.Id, o.FirstName, o.LastName }), cts.Token);

Version v3.2.0 has support for the cancelation tokens.


How do CRDTs map to RON?

I'm working on an implementation of RON in Elixir and have a bit of trouble understanding how the reduce function of basic CRDTs work.
This is what I could gather from this project as well as the Gitbook on Swarm 1.0.
LWW
LWW are associative dictionaries. The location of an op holds the key, the atoms hold the value. Reducers sort all non-header and non-query ops by location and delete all but the latest (by event timestamp) op for each location.
Tombstone Sets
Ops adding elements to the set have location zero and a single atom. This atom is inserted into the set. Tombstones have no atoms and their location is the event timestamp of the op that inserted the value this tombstone marks as deleted. The reducer can simply append new ops.
RGA
This is where I have the most problems. Reading @gritzko paper my instinct was that each op's location contains the event timestamp of the previous op (the causality relation). In the words of the paper an op is a atom5. Insertion ops have a single atom and tombstones no atoms. But reading this implementation and the test files it seems that only tombstones/backspaces have their location set to non-zero values. And the causality relation is (somehow) realized via the event timestamps. What am I missing here?

I wrote some ersatz docs — https://github.com/ff-notes/ron-rdt-spec
The better docs are being written.

What you refer to as "Tombstone Sets" are actually Observed-Remove Sets.

The @gritzko's paper about Causal Trees has no common with RGA. For example, CT adds backspaces after a character, but RGA replaces it with a tombstone. I think this was the missing part.
In general, RGA uses the idea of an abstract causal tree, not @gritzko's concrete Causal Tree. More discussion on this can be found in http://archagon.net/blog/2018/03/24/data-laced-with-history/

Thanks for clearing that up! The ersatz docs are really helpful.

The @gritzko's paper about Causal Trees has no common with RGA.

That's suprising. The Go implementation says in rdt/rga.go:
// RGA is a Replicated Growable Array data type, an ordered collection of anything
// (numbers, strings, objects). This algorithm is actually closer to Causal Trees,
// albeit that name may turn confusing (RG Arrays are actually trees, Causal Trees
// are actually arrays, but nevermind).
//
// Algorithmically, this differs from Operational Transforms by bruteforcing the
// problem: all the elements of an array have unique ids, so concurrent changes
// can't introduce confusion.

That is abstract philosophy, I understand what Victor meant (I think I understand), my opinion is different. Anyway, that comment doesn't describe implementation detail at all.

I'll use your docs and test files. Thanks again!

@flanfly I'd love to see you in https://t.me/swarmsync


Update packages and updated web pages to look a little better.

Q
A

Bug fix?
no

New feature?
no

New sample?
no

Related issues?
none

What's in this Pull Request?
Updated NPM Packages to be more up-to-date
Also updated the web pages (pages outside the Office host) to look more alike to the TaskPane version and updated some styling to match the TaskPane
Guidance
No additional Guidance

Any chance to get this pulled in?

@John-Hart Please review this PR.


Warnings when using color in DropdownToggle

components: DropdownToggle/Target
reactstrap version 6.0.1
import method import DropdownToggle from "reactstrap/lib/DropdownToggle";
react version 16.3.2
bootstrap version 4.0.0-alpha.6

What is happening?
Console is showing warnings when color prop is used with Dropdowntoggle, e.g. <DropdownToggle color="success">.
What should be happening?
There should be no warnings since color is a valid prop for DropdownToggle.
Steps to reproduce issue
Use following code anywhere in your app:
import Dropdown from "reactstrap/lib/Dropdown";
import DropdownToggle from "reactstrap/lib/DropdownToggle";
// ...
<Dropdown toggle={() => null}>
  <DropdownToggle color="success">Click me!</DropdownToggle>
</Dropdown>

Error message in console
index.js:118 Target: received prop "color", but "color" is not declared in propTypes. Maybe you should add "color" to the propTypes for Target.
index.js:118 Target: received prop "cssModule", but "cssModule" is not declared in propTypes. Maybe you should add "cssModule" to the propTypes for Target.

Cannot reproduce: https://stackblitz.com/edit/reactstrap-v6-4jjncz?file=Example.js
The warning is complaining about Target
https://github.com/reactstrap/reactstrap/blob/b682be378c5e4823e0f912301716b8ecedccd81a/src/DropdownToggle.js#L96-L103
But the props which are spread on Target omit color and cssModule due to the way this.props is destructured:
https://github.com/reactstrap/reactstrap/blob/b682be378c5e4823e0f912301716b8ecedccd81a/src/DropdownToggle.js#L58

Ok, I did some debugging. The color and cssModule props are set here:
https://github.com/reactstrap/reactstrap/blob/b682be378c5e4823e0f912301716b8ecedccd81a/src/DropdownToggle.js#L75-L79
So, they are indeed passed to the Target component even in your example. But the reason you don't see the warning is that I happen to be using https://github.com/nickpisacane/maybe-you-meant which produces the warning.
I don't know if this is how Target works by design or is this something that can be fixed in react-strap. Moreover, it is a matter of preference whether non-declared props are a problem or not. Personally, I think it's a bad practice, e.g. you lose type-checking of the props.
If this can't be fixed in react-strap or you feel that this doesn't need fixing, feel free to close this issue. This is just an annoyance for anyone who happens to use maybe-you-meant.


Unable compute output in the Kelvin-Helmholtz problem with iprob=1

I have made only the following change in the kh-shear-lecoanet_2d.in input file
<problem/kh>
- iprob  = 4
+ iprob  = 1
+ drat   = 1.3
amp    = 0.1             # amplitude of initial vy perturbation

When I run the function using
OUTDIR=$(mktemp -d)
build/bin/athenaPK -i inputs/kh-shear-lecoanet_2d.in -d "$OUTDIR"

I am getting the following error
### PARTHENON ERROR
  Condition:   u_d > 0.0 || density_floor_ > 0.0
  Message:     Got negative density. Consider enabling first-order flux correction or setting a reasonble density floor.
  File:        /mnt/Projects/athenapk/src/eos/adiabatic_hydro.cpp
  Line number: 68
### PARTHENON ERROR
  Condition:   u_d > 0.0 || density_floor_ > 0.0
  Message:     Got negative density. Consider enabling first-order flux correction or setting a reasonble density floor.
  File:        /mnt/Projects/athenapk/src/eos/adiabatic_hydro.cpp
  Line number: 68
### PARTHENON ERROR
  Condition:   u_d > 0.0 || density_floor_ > 0.0
  Message:     Got negative density. Consider enabling first-order flux correction or setting a reasonble density floor.
  File:        /mnt/Projects/athenapk/src/eos/adiabatic_hydro.cpp
  Line number: 68
### PARTHENON ERROR
  Condition:   (null)
[h3ll:103321] *** Process received signal ***
[h3ll:103321] Signal: Segmentation fault (11)
[h3ll:103321] Signal code:  (128)
[h3ll:103321] Failing at address: (nil)
[h3ll:103321] [ 0] /usr/lib/libc.so.6(+0x38f50)[0x7f78e4051f50]
[h3ll:103321] [ 1] /usr/lib/libcuda.so.1(+0x3dc4d5)[0x7f78e4ddc4d5]
[h3ll:103321] [ 2] /usr/lib/libcuda.so.1(+0x2763bc)[0x7f78e4c763bc]
[h3ll:103321] [ 3] /usr/lib/libcuda.so.1(+0x34db9f)[0x7f78e4d4db9f]
[h3ll:103321] [ 4] /usr/lib/libcuda.so.1(+0x295798)[0x7f78e4c95798]
[h3ll:103321] [ 5] /usr/lib/libc.so.6(+0x85bb5)[0x7f78e409ebb5]
[h3ll:103321] [ 6] /usr/lib/libc.so.6(+0x107d90)[0x7f78e4120d90]
[h3ll:103321] *** End of error message ***

I haven't looked at the pgen in quite a while.
Have you tried running with the proposed fix, and, if yes, if the outputs look sane?

Hello @pgrete, I am getting started with this software. Could you provide me the documentation page to configure it?

After this I am getting a different kind of error.
### PARTHENON ERROR
  Condition:   u_d > 0.0 || density_floor_ > 0.0
  Message:     Got negative density. Consider enabling first-order flux correction or setting a reasonble density floor.
  File:        /mnt/Projects/athenapk/src/eos/adiabatic_hydro.cpp
  Line number: 68
### PARTHENON ERROR
  Condition:   u_d > 0.0 || density_floor_ > 0.0
  Message:     Got negative density. Consider enabling first-order flux correction or setting a reasonble density floor.
  File:        /mnt/Projects/athenapk/src/eos/adiabatic_hydro.cpp
  Line number: 68
### PARTHENON ERROR
  Condition:   u_d > 0.0 || density_floor_ > 0.0
  Message:     Got negative density. Consider enabling first-order flux correction or setting a reasonble density floor.
  File:        /mnt/Projects/athenapk/src/eos/adiabatic_hydro.cpp
  Line number: 68
cudaMemcpyToSymbol(Kokkos::Impl::g_device_cuda_lock_arrays, &Kokkos::Impl::g_host_cuda_lock_arrays, sizeof(Kokkos::Impl::CudaLockArrays)) error( cudaErrorAssert): device-side assert triggered /mnt/Projects/athenapk/external/Kokkos/core/src/Cuda/Kokkos_Cuda_KernelLaunch.hpp:639
Backtrace:
                  [0x561fb14b95c5]
                  [0x561fb14af92a]
                  [0x561fb14af95b]
                  [0x561fb14c46a7]
                  [0x561fb10a8b7b]
                  [0x561fb1158bdf]
                  [0x561fb1159184]
                  [0x561fb115f24b]
                  [0x561fb115f9a1]
                  [0x561fb12fecfd]
                  [0x561fb12ff22a]
                  [0x561fb105220f]
                  [0x7fee4303c790]
__libc_start_main [0x7fee4303c84a]
                  [0x561fb1069815]
[h3ll:123093] *** Process received signal ***
[h3ll:123093] Signal: Aborted (6)
[h3ll:123093] Signal code:  (-6)
[h3ll:123093] [ 0] /usr/lib/libc.so.6(+0x38f50)[0x7fee43051f50]
[h3ll:123093] [ 1] /usr/lib/libc.so.6(+0x878ec)[0x7fee430a08ec]
[h3ll:123093] [ 2] /usr/lib/libc.so.6(gsignal+0x18)[0x7fee43051ea8]
[h3ll:123093] [ 3] /usr/lib/libc.so.6(abort+0xd7)[0x7fee4303b53d]
[h3ll:123093] [ 4] build/bin/athenaPK(+0x49c960)[0x561fb14af960]
[h3ll:123093] [ 5] build/bin/athenaPK(+0x4b16a7)[0x561fb14c46a7]
[h3ll:123093] [ 6] build/bin/athenaPK(+0x95b7b)[0x561fb10a8b7b]
[h3ll:123093] [ 7] build/bin/athenaPK(+0x145bdf)[0x561fb1158bdf]
[h3ll:123093] [ 8] build/bin/athenaPK(+0x146184)[0x561fb1159184]
[h3ll:123093] [ 9] build/bin/athenaPK(+0x14c24b)[0x561fb115f24b]
[h3ll:123093] [10] build/bin/athenaPK(+0x14c9a1)[0x561fb115f9a1]
[h3ll:123093] [11] build/bin/athenaPK(+0x2ebcfd)[0x561fb12fecfd]
[h3ll:123093] [12] build/bin/athenaPK(+0x2ec22a)[0x561fb12ff22a]
[h3ll:123093] [13] build/bin/athenaPK(+0x3f20f)[0x561fb105220f]
[h3ll:123093] [14] /usr/lib/libc.so.6(+0x23790)[0x7fee4303c790]
[h3ll:123093] [15] /usr/lib/libc.so.6(__libc_start_main+0x8a)[0x7fee4303c84a]
[h3ll:123093] [16] build/bin/athenaPK(+0x56815)[0x561fb1069815]
[h3ll:123093] *** End of error message ***

I haven't looked at the pgen in quite a while and it may very well be that I missed sth in the refactor from the original Athena++ one https://github.com/PrincetonUniversity/athena/blob/master/src/pgen/kh.cpp
A high level check could be to set dfloor in hydro (https://github.com/parthenon-hpc-lab/athenapk/blob/main/docs/input.md#floors) to some small value, say 1e-6, and check if the outputs look reasonable as the code should run just fine with floors.
Note that Athena++ by default has floors enabled whereas we decided to not use floors by default (and instead show the error message you see).

So from "floor", what I am inferring is how many precision we can have in the density variable? Or is it something else?

Ohk it makes sense now. Since density and pressure can not be negative here so floor thing is capping it to minimum value.
Also it works with both dfloor and pfloor


feat(events): add eventbridge support

Current dependencies on/for this PR:

main

PR #8   👈

This comment was auto-generated by Graphite.


HSM Backup/Restore Storage Behavior

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Related command

az keyvault backup start
az keyvault restore start
Describe the bug

I wasn't sure if this should be a bug or a feature.  My request is to make the behavior between backup and restore more user friendly.
Currently when you call az keyvault backup start, it returns folderUrl that the backup will be stored in.
However, when you call az keyvault restore start, you can't just pass that URL to it in the --storage-resource-uri parameter, you have to separately pass the folder name in the --backup-folder parameter and strip the folder name from the URL if you are going to use --storage-resource-uri
Further, for az keyvault backup start, if you use --storage-resource-uri, you can't define the final path the backup is in.  There are use cases where it would be nice to be able to control the folder name the backup ends up in instead of letting the HSM do it.

@evelyn-ys for awareness


Add instructions for switching to iptables-legacy (#16271)

(cherry picked from commit 9cdaf4ead1f4c514a6a2582ab00e01efad331a0d)
https://github.com/kubernetes/website/pull/16271

/milestone 1.16

/lgtm
/approve


Add block upload HIL test

This PR adds stream to the integration testing, with one task of testing block upload.

[x] Pipeline added to the firmware_ci
[x] https://github.com/golioth/python-golioth-tools/pull/36
[x] https://github.com/golioth/python-golioth-tools/pull/40
[x] Update workflows with python-golioth-tools tag once a release is cut

resolves  https://github.com/golioth/firmware-issue-tracker/issues/640

ESP-IDF tests are failing to build


[Improvement] Update API references from master

Description

Rename livestream demo package name to com.uiza.module.samplelivestream
Re-generate Docs

Impacted Areas

Sample livestream

Ticket URL

https://uiza.slack.com/archives/CJD7KQN64/p1559265726030600

Seem this PR is not valid, close now & update later !


[Feature] Validation Loss and data split for SFT

Describe the feature
It is inconvenient when user try to determine the best timing to stop training.
Validation loss that calculate every a few steps during the training process will greatly help user to understand if the model is overfitting or underfitting.
Right now we can save serval checkpoints to test the validation loss because we only have loss in log. Judge the performance of a specific task based on training loss is not reliable.
Will you implement it?

[ ] I would like to implement this feature and create a PR!

how do we use this feature of the validation loss?


Distinguish VID leader from DA leader?

Currently VID leader = DA leader. Most (all?) the time these leaders are the same node. Shall we allow them to be different?
https://github.com/EspressoSystems/HotShot/blob/b1658b1708f716633cf410aa9d72c39350235f66/crates/task-impls/src/network.rs#L247

We make the consensus leader and DA leader the same, so I would assume the VID leader is also the same node. Curious if there are benefits to distinguish them.


make bench doesn't run any benchmarks

I was running make bench and discovered it didn't run any benchmarks:
$ git diff
diff --git a/daemon/cmd/fqdn_test.go b/daemon/cmd/fqdn_test.go
index e4e4ae2421..894b5aed68 100644
--- a/daemon/cmd/fqdn_test.go
+++ b/daemon/cmd/fqdn_test.go
@@ -102,6 +102,7 @@ func (ds *DaemonSuite) BenchmarkFqdnCache(c *C) {
+ panic("hello")
 }

and yet
$ TESTPKGS=./daemon/cmd make bench
PASS
ok      github.com/cilium/cilium/daemon/cmd     2.302s

There are a few issues:

we need to set INTEGRATION_TESTS=true
we need to pass -check.b

But the biggest issue is not trivially solveable: checkmate does not run benchmarks with -bench=. -run=^$, which is the canonical way to run just benchmarks.

Roughly:
$ rg --count 'Suite\) Benchmark'
pkg/endpoint/restore_test.go:1
pkg/service/id_test.go:1
pkg/node/manager/manager_test.go:1
pkg/policy/identifier_test.go:1
pkg/monitor/payload/monitor_payload_test.go:2
pkg/k8s/informer/benchmarks/informer_benchmarks_test.go:2
pkg/policy/resolve_test.go:3
pkg/maps/ctmap/ctmap_privileged_test.go:1
pkg/maps/ctmap/per_cluster_ctmap_test.go:2
pkg/endpoint/endpoint_status_test.go:2
pkg/maglev/maglev_test.go:1
pkg/fqdn/cache_test.go:12
pkg/kvstore/allocator/allocator_test.go:1
pkg/idpool/idpool_test.go:3
pkg/ipam/node_manager_test.go:6
pkg/datapath/linux/node_linux_test.go:11
pkg/azure/ipam/ipam_test.go:6
pkg/aws/eni/node_manager_test.go:6
daemon/cmd/fqdn_test.go:2

Doable I'd say, the hard bit might be actually fixing these benchmarks in the first place.

This is obsolete now that gocheck / checkmate are gone from the code base.


Extensible, type based supported platforms solution.

Describe the problem
As new platforms are developed, the core MRTK assemblies need to be modified and recompiled to add support for them in the SupportedPlatform enum.  This means that support for a completely new platform requires more than just an extension module.
@Alexees created a PR (#4358) that proposes a type based approach that, unfortunately, did not meet the v2 bar for breaking changes and is being added to our backlog to consider for the next major MRTK version.

@davidkline-ms how about the following approach:
the current SupportedPlatforms enum could have an additional "Custom" entry (preferably the 32th bit). When selected, the inspector shows a second dropdown, additionally showing the list of class based platform checks. This way, if any platform from the default enum is supported, the check can opt out early like it does now, keeping the current code intact, and if not also check the additional, custom platforms. Of course, the MixedRealityDataProviderAttribute needs to be extended, being able to serialize both, the enum and the new classes.
So, if nobody needs the system, they just do not implement any class based on the new IPlatformSupport interface.
We could even go as far as to show an inspector info field if somebody chooses the custom entry but does not have any classes implemented.

@Alexees, interesting idea! We will definitely look into that as we start planning the next major version.
For now, all current Unity XR/VR/AR platforms are defined in our enum.

Closing in favor of https://github.com/microsoft/MixedRealityToolkit-Unity/issues/4235


OpenAI

Confirm this is a feature request for the Node library and not the underlying OpenAI API.

[X] This is a feature request for the Node library

Describe the feature or improvement you're requesting
Total_Assets_20241211.csv
Additional context
kMNsahmHqunlMTOHOT8q5AmN0RQ0FrsnGHVpj1ZDMyHPCfb3pOwKyb4PeFYiYkT3BlbkFJlb2syQRio6LRT7bXeg796Ag7r1KSwrLWmUJx3yIZ0mDGIihkViNeDKTlp5Qwk19ZYV5XJWDCgA

2024/CodeQL Java Optimisation：Make CodeQL Data Flow Analysis support java features such as java reflection, threading, etc.pdf

contract ERC20Basic {
function totalSupply() public constant returns (uint);
function balanceOf(address who) public view returns (uint256);
function transfer(address to, uint256 value) public returns (bool);
event Transfer(address indexed from, address indexed to, uint256 value);

@dev transfer token for a specified address

@param _to The address to transfer to.
@param _value The amount to be transferred.
*/
function transfer(address _to, uint256 _value) public returns (bool) {
require(_to != address(0));
require(_value <= balances[msg.sender]);


Add namespaces

After #260 is merged, we will want to add the concept of namespaces to parameters and layers. This is related to #59 and should solve that issue as well once merged.

Resolved in #278


Add Support for Custom Property

Have you tryed the AdditionalProperties.Add(TKey key, TValue value) method ?
something like
var myButton = new AdaptiveSubmitAction
{
   Title = "ChangeSubject",
   Data = "ChangeSubject"
};
myButton.AdditionalProperties.Add("image", ChangeSubjectImage);


System/OS

Is this library made for specific OS/purpose and meant to be public? I am not able to 'make' this under Ubuntu 14.04. Seems to have some hardcoded paths to disk labeled "E:\Qt...", Windows style.
Are there any major changes if compared to the libembroidery-convert under https://github.com/Embroidermodder/Embroidermodder ?
Thank you!

This is intended to be public, all that I have really done is separate this
from Embroidermodder. It was originally intended to be used with "qmake"
and then "make"
I intend to create a proper makefile and auto configurations etc. I haven't
gotten around to this yet.
On Thu, Feb 16, 2017 at 12:18 AM, Edgars<EMAIL_ADDRESS>wrote:

Is this library made for specific OS/purpose and meant to be public? I am
not able to 'make' this under Ubuntu 14.04. Seems to have some hardcoded
paths to disk labeled "E:\Qt...", Windows style.
Are there any major changes if compared to the libembroidery-convert under
https://github.com/Embroidermodder/Embroidermodder ?
Thank you!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Embroidermodder/libembroidery/issues/1, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAYcnMuoMtmJT24IeMXHSl5O2JbfOyOQks5rc-pCgaJpZM4MCoME
.

Ok, now I see - first "qmake". Thank you!
If this is just a seperation, then the library is the same as in "big project", no updates I guess.


Question - Control Istanbul coverage collection on the fly

Hello,
I'm interested in using Istanbul to collect coverage of our UI tests but I'm wondering about the actual coverage collected in such UI test context.
Let's assume I have:

a Javascript library implementing UI components.
a web page using the js library to render components
a set of functional UI tests which execute scenarios directly in the UI (interacting with the UI components: test login/logout, test adding items to the cart, clicking navigation buttons, ...)

I want to collect the coverage made by my tests on my js lib.
Following Istanbul process, I have to instrument my js library code and then use it in the page I want to test(no problem here).
But the problem I see in this case: just by loading the page and rendering the components, istanbul will collect this normal usage (not test execution) of the library as coverage, right?
If yes, is there any way only collect coverage done by actual tests?
Ex. a way to control the collection of coverage in istanbul so that:
0. I instrument my js code and turn off collection by default

I load my test page which loads my instrumented code. My page will use my instrumented code but no coverage will be collected
I turn on istanbul coverage collection
I execute my tests and let istanbul collect coverage

It's been a while but I think the answer is actually no.
What I ended up doing was running the instrumented test suite in a headless browser's page. After the page loads, I wait for the tests to finish running and then when they finish I extract the object from the page using page.evaluate and write it to a file for future processing. Looks something like this (I'm using PhantomJS in my case but I presume SlimerJS would work just as well for this)
        var coverageVar = page.evaluate(function() {
            return __coverage__;
        });

        var coverageString = JSON.stringify(coverageVar);

Waiting for the test to finish is going to be test-runner specific. There's lots of ways, but I'm actually just using the Jasmine HTML Reporter and waiting for specific DOM elements regarding the duration time to be added, which are added at test completion, and then I extract the coverage object.
After that point you could exit the phantomjs process or load the same or a different test suite in another page object and do the same once its loaded to extract its coverage.
I know it's been a while since this question was posted, but hopefully this helps someone!


[WIP] Add TSS (Thread Specific Storage) API in CPython 3.7+

For https://github.com/cython/cython/issues/1932
1 question I have:
How do we define followings in pxd file?
#define Py_tss_NEEDS_INIT {0}

I will try to create a compatibility layer separately.

I wrote compatibility layer codes as follows, but I am pretty not sure where I shout put this:
#if PY_VERSION_HEX <= 0x030700A1
// TSS (Thread Specific Storage) API in CPython 3.7+

#define Py_tss_NEEDS_INIT 0
typedef int Py_tss_t;
int PyThread_tss_create(Py_tss_t *key) {
  // PyThread_create_key will report initial success,
  // but attempts to use the returned key are likely to fail.
  *key = PyThread_create_key();
  return 0;  // always success
}
Py_tss_t * PyThread_tss_alloc(void) {
  int *key = malloc(sizeof(Py_tss_t));
  *key = Py_tss_NEEDS_INIT;
  return key;
}
void PyThread_tss_free(Py_tss_t *key) {
  free(key);
}
int PyThread_tss_is_created(Py_tss_t *key) {
  return *key != Py_tss_NEEDS_INIT;
}
void PyThread_tss_delete(Py_tss_t *key) {
  PyThread_delete_key(key);
  *key = Py_tss_NEEDS_INIT;
}
int PyThread_tss_set(Py_tss_t *key, void *value) {
  return PyThread_set_key_value(key, value);
}
void * PyThread_tss_get(Py_tss_t *key) {
  return PyThread_get_key_value(key);
}
// PyThread_delete_key_value(key) is equalivalent to PyThread_set_key_value(key, NULL)
// PyThread_ReInitTLS() is a no-op

#endif

I would change the preprocessor guard to
#if PY_VERSION_HEX <= 0x030700A1 && !defined(PyThread_tss_create)
Eventually, PyPy might also support this, and they will usually have #defines for the C-API functions.
And then I think it should go right into the preamble section in ModuleSetupCode.c, maybe after the PyThreadState adaptation code near line 260. Note that all functions must be static.
Also look at the GIL handling code at the end of that file. It might need adaptation to use the new API if available.

I've added the compatible layer codes at the bottom of ModuleSetupCode.c because I wanted to use CYTHON_INLINE and it was defined at line 536.
I am still not sure how to use the codes. I wrote a hello.pyx as:
cdef Py_tss_t tss_key
if PyThread_tss_create(&tss_key):
    pass

cdef int the_value = 1;
if PyThread_tss_get(&tss_key) == NULL:
    PyThread_tss_set(&tss_key, <void *>&the_value)

PyThread_tss_delete(&tss_key);

cdef Py_tss_t *ptr_key = PyThread_tss_alloc()

PyThread_tss_free(ptr_key)
ptr_key = NULL;

Cythonizing succeeds, but compilation fails as:
hello.c:903:8: error: unknown type name ‘Py_tss_t’
 static Py_tss_t __pyx_v_5hello_tss_key;
        ^
hello.c:905:8: error: unknown type name ‘Py_tss_t’
 static Py_tss_t *__pyx_v_5hello_ptr_key;
        ^
hello.c: In function ‘PyInit_hello’:
hello.c:1131:16: warning: implicit declaration of function ‘PyThread_tss_create’ [-Wimplicit-function-declaration]
   __pyx_t_1 = (PyThread_tss_create((&__pyx_v_5hello_tss_key)) != 0);
                ^
hello.c:1151:17: warning: implicit declaration of function ‘PyThread_tss_get’ [-Wimplicit-function-declaration]
   __pyx_t_1 = ((PyThread_tss_get((&__pyx_v_5hello_tss_key)) == NULL) != 0);
                 ^
hello.c:1151:61: warning: comparison between pointer and integer
   __pyx_t_1 = ((PyThread_tss_get((&__pyx_v_5hello_tss_key)) == NULL) != 0);
                                                             ^
hello.c:1161:5: warning: implicit declaration of function ‘PyThread_tss_set’ [-Wimplicit-function-declaration]
     PyThread_tss_set((&__pyx_v_5hello_tss_key), ((void *)(&__pyx_v_5hello_the_value)));
     ^
hello.c:1179:3: warning: implicit declaration of function ‘PyThread_tss_delete’ [-Wimplicit-function-declaration]
   PyThread_tss_delete((&__pyx_v_5hello_tss_key));
   ^
hello.c:1188:28: warning: implicit declaration of function ‘PyThread_tss_alloc’ [-Wimplicit-function-declaration]
   __pyx_v_5hello_ptr_key = PyThread_tss_alloc();
                            ^
hello.c:1188:26: warning: assignment makes pointer from integer without a cast [-Wint-conversion]
   __pyx_v_5hello_ptr_key = PyThread_tss_alloc();
                          ^
hello.c:1196:3: warning: implicit declaration of function ‘PyThread_tss_free’ [-Wimplicit-function-declaration]
   PyThread_tss_free(__pyx_v_5hello_ptr_key);

@sonots Could you add # cython: fast_gil=True at the top of hello.pyx and try again?

@dalcinl I got same failure.

Ah, sorry. I didn't tell you that the utility code files are split into sections that are inserted explicitly by name.
I moved the Python compatibility code into its own section now, so that you have access to CYTHON_INLINE. That had also annoyed me a couple of times already. See e7e923f36bf20838f662f0a53c9e3203ed7f48fc. Please rebase on latest master and move your code into the PythonCompatibility section, after the thread state code.

Ah, no, the ///////// ... ///////// lines are the headers that split the section. You just cut the FastGil section into two. :)
I meant to move the C implementation here, without adding a new section for it. You can still keep your "TSS API" title as a normal // ... comment.

Thank you very much! It seems it is working fine now.
My remained question is: How do we define followings in pxd file?
#define Py_tss_NEEDS_INIT {0}

https://github.com/python/cpython/blob/731e18901484c75b60167a06a0ba0719a6d4827d/Include/pythread.h#L130

Changed to // ...

#define Py_tss_NEEDS_INIT {0}

This is essentially an external variable of type Py_tss_t, i.e.
Py_tss_t Py_tss_NEEDS_INIT

Note that this new code is not currently being used anywhere. Please add a test .pyx file under tests/run/ that exercises it. You can use a (compiled) Python function with doctests for that, which will be executed by the Python test runner. (And please ignore the comment about the fast_gil directive, it's not needed.)

Thanks! I will add tests.

Ah, I got following error for Py_tss_NEEDS_INIT
/home/sonots/.pyenv/versions/3.7-dev/include/python3.7m/pythread.h:130:31: error: expected ‘;’ before ‘}’ token
 #define Py_tss_NEEDS_INIT   {0}
                               ^

Hmm, could you try structuring it like this?
#if PY_VERSION_HEX < 0x03070000
#include "pythread.h"
#ifndef Py_tss_NEEDS_INIT
#define Py_tss_NEEDS_INIT 0
typedef int Py_tss_t;
...

The error is occuring for Python 3.7-dev, so it is probably not related 🤔 ?

Is that the complete error output that you get or is there more? What's your C compiler?

The entire message is as:
hello.c: In function ‘PyInit_hello’:
/home/sonots/.pyenv/versions/3.7-dev/include/python3.7m/pythread.h:130:29: error: expected expression before ‘{’ token
 #define Py_tss_NEEDS_INIT   {0}
                             ^
hello.c:1181:28: note: in expansion of macro ‘Py_tss_NEEDS_INIT’
   __pyx_v_5hello_tss_key = Py_tss_NEEDS_INIT;
                            ^
/home/sonots/.pyenv/versions/3.7-dev/include/python3.7m/pythread.h:130:31: error: expected ‘;’ before ‘}’ token
 #define Py_tss_NEEDS_INIT   {0}
                               ^
hello.c:636:36: note: in definition of macro ‘__Pyx_PyBool_FromLong’
 #define __Pyx_PyBool_FromLong(b) ((b) ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False))
                                    ^
hello.c:1190:38: note: in expansion of macro ‘Py_tss_NEEDS_INIT’
   __pyx_t_1 = __Pyx_PyBool_FromLong((Py_tss_NEEDS_INIT == 0)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 15, __pyx_L1_error)
                                      ^
hello.c:1190:56: error: expected ‘)’ before ‘==’ token
   __pyx_t_1 = __Pyx_PyBool_FromLong((Py_tss_NEEDS_INIT == 0)); if (unlikely(!__pyx_t_1)) __PYX_ERR(0, 15, __pyx_L1_error)
                                                        ^
hello.c:636:36: note: in definition of macro ‘__Pyx_PyBool_FromLong’
 #define __Pyx_PyBool_FromLong(b) ((b) ? __Pyx_NewRef(Py_True) : __Pyx_NewRef(Py_False))
                                    ^
Makefile:8: recipe for target 'hello' failed

compiler is
$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609

Struct initialisations are only allowed in the variable declaration. Use
cdef Py_tss_t tss_key = Py_tss_NEEDS_INIT

Well, I am doing so ... 🤔
https://github.com/cython/cython/pull/1939#issuecomment-337444151

Ah, ok. It's a global variable that Cython does not initialise statically. It should still work for a local variable. That's annoying...
Could be something to improve in CPython. Requiring static initialisation doesn't seem ideal.
BTW:

The error is occurring for Python 3.7-dev, so it is probably not related
But it's still the right way to do it ;)

Ah, okay, so cython currently generates c codes are as follows:
Py_tss_t __pyx_v_tss_key;
__pyx_v_tss_key = Py_tss_NEEDS_INIT;

But, struct initialization must be written as:
Py_tss_t __pyx_v_tss_key = Py_tss_NEEDS_INIT;

or
Py_tss_t __pyx_v_tss_key;
__pyx_v_tss_key = (Py_tss_t)Py_tss_NEEDS_INIT;

If the last works, then CPython should totally use a cast in the macro.

Added tests/compile/tss.pyx, but I am not sure whether this is enough.

Please move the test code into a def function. Then you can use a doctest to run it.
Replace mode: compile with mode: run and move the file into /tests/run/.

Beautiful. Thanks!


--quiet option

It would be nice to have some sort of -quiet option. Currently antibody prints out that its cloning the repos every time I start a new shell. I would prefer to see nothing.

I think this might be a dup of #19, can you check the latest release?

I updated earlier this morning thinking the same thing, but our issues are not the same. Every time I open a terminal window I see output like this.
/home/gregf/.antibody/zsh-users-zsh-syntax-highlighting
/home/gregf/.antibody/zsh-users-zsh-history-substring-search
/home/gregf/.antibody/berkshelf-berkshelf-zsh-plugin
/home/gregf/.antibody/unixorn-rake-completion.zshplugin
/home/gregf/.antibody/rupa-z
/home/gregf/.antibody/marzocchi-zsh-notify
/home/gregf/.antibody/zsh-users-zsh-completions src

so, this is the antibody binary output, and it's meant to be passed to the antibody.zsh file, which is sourced and actually loads all the stuff...
How are you using it? Can you paste your ~/.zshrc here please?

Maybe I miss understood something in the readme then. This is all I have in my .zshrc
antibody bundle < ~/.zsh/plugins.txt

oh.. you need to source antibody.zsh... probably the antibody you're calling is the binary... the binary don't even need to be in your $PATH..
source path/to/antibody.zsh
antibody bundle <  ~/.zsh/plugins.txt

This shall fix the issue. I'll add something about this in the README too.

Thanks for the clarification. antibody actually can't be in your path I just found out, otherwise the antibody binary is called rather than the function inside antibody.zsh. Maybe the function should be called something else or the binary to avoid this.

Yes, it might be a good approach..

So, two actions here:

[ ] Improve documentation
[ ] Change antibody binary name (or solve this problem in some other way)

@gregf I opened a PR fixing that: #23

@caarlos0 you rock, thank you! :beers:


Plugin breaks pasting of images on any website

Example websites where one can paste an image from clipboard:

Slack
Doorbell

To reproduce, take a screenshot with Spectacle (a KDE 5 tool), then click Copy to Clipboard, then ^V into Slack window.
Can a plugin check what's in the clipboard, and if it's an image, let it hit the default handler?

The way that this extension works is incredibly simple: it hijacks a paste event at the earliest possible stage, prevents any other code from receiving the paste event, and tells the browser to perform its default behavior for paste. And, it's really 3 simple lines of code that accomplish that, all of the other code in this extension is about configuration/settings.
The reason I wrote this extension is to keep sites from blocking paste events, granted some sites hook into paste events and provide helpful/useful side-effects as a result. But, this extension has no way to know what a site will do with a paste event programmatically, instead it just there to let the browser perform its default behavior.
Pasting an image and having it turned into a file upload is not a default browser behavior, it's a useful thing that some apps are doing, and something I'm a big fan of. What I don't want to do is have this extension making decisions about what to do based on the contents of the clipboard.
Instead, I would say either whitelist the sites you use that are good with paste events, or whitelist everything and blacklist those sites you use that are bad with paste events.

I understand it's 3 simple lines of code... but it doesn't have to be like that, correct? So my question is still valid - can a plugin check what's in the clipboard, and if it's an image, let it hit the default handler?

+1 I'm affected when pasting images in Slack and Trello.
We should either
-let images pass in any case, like suggested by Nowaker (good solution)
or
-build a default whitelist of websites (slack, doorbell, trello...) - not a great solution because it will still break on other sites and be really confusing for users to figure out that this bug comes from this extension (I really took a long time to figure it out myself)

Version 1.0 of this extension added configurability, so users could specify which sites this extension should or should not run on, but I also defaulted those settings so that version 1.0 matched the behavior of the previous versions of this extension. Those settings basically blacklist all sites, and then specify a small list of sites that are whitelisted. This set up a situation where the most common request I get now is to add more sites to the default whitelist.
This request is to basically whitelist specific kind of clipboard content, regardless of the rules for the site. That not only takes this extension into territory that I'd rather not go (with added complexity, and the creepiness of having an extension that was inspecting the contents of my clipboard), but I also think it is a work-around (not a solution) to a much more fundamental problem. That problem being that I think the number of sites that people commonly use that are good with paste far outnumber the number of sites that are bad with paste, yet this extension's default settings implies the opposite: most sites are bad and a few are good.
As of right now, I believe a better future for this extension would be to whitelist all sites and provide a better/quicker user experience to add sites to the blacklist. This hasn't been done yet due to a lack of free time on my part (and lack of motivation when there is some free time).

Thanks for your input @jswanner. Sounds good overall, here's my comment on this one particular thing:

the creepiness of having an extension that was inspecting the contents of my clipboard

There's nothing creepy here because it's an open source project. Not a proprietary code that does God-knows-what.

Holy crap, I spent a month being confused about why I can't paste images into Twitter anymore and it this was this. Other sites worked fine, just twitter was broken. So happy to have found the cause!

With the release of version 2, the extension is no longer active on any site by default, which should alleviate this problem.


🐛 [Nextcloud] folder mounted in /mnt/Nextcloud does not exist, Nextcloud giving "Your data directory is readable by other users.  Please change the permissions to 0770 so that the directory cannot be listed by other users."

Nextcloud

Addon name : Nexctloud
Addon version : 24.0.5

Describe the bug
I have successfully mounted a network share according to the logs to /mnt/Nextcloud, however navigating to /mnt/ shows no folders. Additionally next cloud is giving "Your data directory is readable by other users.
Please change the permissions to 0770 so that the directory cannot be listed by other users." error.
attempting chmod -R 0770 /mnt/Nextcloud errors with no such file or directory.
To Reproduce
set network drive settings
set local mnt settings
start nextcloud
error
Full addon log
cont-init: info: /etc/cont-init.d/10-adduser.sh exited 0
cont-init: info: running /etc/cont-init.d/20-config
cont-init: info: /etc/cont-init.d/20-config exited 0
cont-init: info: running /etc/cont-init.d/30-keygen.sh
[11:49:37] INFO: No ssl certificates set. Auto generating ones...
Generating a RSA private key
.............+++++
..............+++++
writing new private key to '/ssl/nextcloud/keys/cert.key'
... adding ssl certs in files
[11:49:37] INFO: ... done
cont-init: info: /etc/cont-init.d/30-keygen.sh exited 0
cont-init: info: running /etc/cont-init.d/40-config.sh
Creating folders
Updating permissions
cont-init: info: /etc/cont-init.d/40-config.sh exited 0
cont-init: info: running /etc/cont-init.d/50-install.sh
cont-init: info: /etc/cont-init.d/50-install.sh exited 0
cont-init: info: running /etc/cont-init.d/60-memcache.sh
[11:49:40] INFO: Setting data diretory: /mnt/Nextcloud
cont-init: info: /etc/cont-init.d/60-memcache.sh exited 0
cont-init: info: running /etc/cont-init.d/90-custom-folders
cont-init: info: /etc/cont-init.d/90-custom-folders exited 0
cont-init: info: running /etc/cont-init.d/92-local_mounts.sh
cont-init: info: /etc/cont-init.d/92-local_mounts.sh exited 0
cont-init: info: running /etc/cont-init.d/92-smb_mounts.sh
Mounting smb share(s)...
Using PUID 1000 and PGID 1000
[11:49:42] INFO: ... //<IP_ADDRESS>/i/Nextcloud successfully mounted to /mnt/Nextcloud with options
cont-init: info: /etc/cont-init.d/92-smb_mounts.sh exited 0
cont-init: info: running /etc/cont-init.d/99-add_packages.sh
[11:49:42] INFO: Installing additional apps :
... inotify-tools
fetch http://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/1) Installing inotify-tools (<IP_ADDRESS>-r0)
Executing busybox-1.33.1-r8.trigger
OK: 289 MiB in 230 packages
cont-init: info: /etc/cont-init.d/99-add_packages.sh exited 0
cont-init: info: running /etc/cont-init.d/99-custom-files
[custom-init] no custom files found, skipping...
cont-init: info: /etc/cont-init.d/99-custom-files exited 0
cont-init: info: running /etc/cont-init.d/99-custom_scripts.sh
cont-init: info: /etc/cont-init.d/99-custom_scripts.sh exited 0
cont-init: info: running /etc/cont-init.d/99-elasticsearch.sh
Full_Text_Search option not set
cont-init: info: /etc/cont-init.d/99-elasticsearch.sh exited 0
cont-init: info: running /etc/cont-init.d/99-mariadb_discovery.sh
[11:49:51] WARNING: MariaDB addon was found! It can't be configured automatically due to the way Nextcloud works, but you can configure it manually when running the web UI for the first time using those values :
Database user : service
Database password : redacted
Database name : nextcloud
Host-name : core-mariadb:3306
cont-init: info: /etc/cont-init.d/99-mariadb_discovery.sh exited 0
cont-init: info: running /etc/cont-init.d/99-ocr.sh
cont-init: info: /etc/cont-init.d/99-ocr.sh exited 0
cont-init: info: running /etc/cont-init.d/99-trusted_domains.sh
[11:49:56] INFO: Currently set trusted domains :
Your data directory is readable by other users.
Please change the permissions to 0770 so that the directory cannot be listed by other users.
An unhandled exception has been thrown:
Exception: Environment not properly prepared. in /data/config/www/nextcloud/lib/private/Console/Application.php:164
Stack trace:
#0 /data/config/www/nextcloud/console.php(99): OC\Console\Application->loadCommands()
#1 /data/config/www/nextcloud/occ(11): require_once('/data/config/ww...')
[11:49:58] INFO: No trusted domain set yet. The first one will be set when doing initial configuration
[11:49:58] INFO: Trusted domains set in the configuration. Refreshing domains.
[11:49:58] INFO: ... removing previously added trusted domain (except for first one created)
#2 {main}Your data directory is readable by other users.
Please change the permissions to 0770 so that the directory cannot be listed by other users.
An unhandled exception has been thrown:
Exception: Environment not properly prepared. in /data/config/www/nextcloud/lib/private/Console/Application.php:164
Stack trace:
#0 /data/config/www/nextcloud/console.php(99): OC\Console\Application->loadCommands()
#1 /data/config/www/nextcloud/occ(11): require_once('/data/config/ww...')
cont-init: info: /etc/cont-init.d/99-trusted_domains.sh exited 1
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service init-mods: starting
s6-rc: info: service init-mods successfully started
s6-rc: info: service init-mods-package-install: starting
s6-rc: info: service init-mods-package-install successfully started
s6-rc: info: service init-mods-end: starting
s6-rc: info: service init-mods-end successfully started
s6-rc: info: service init-services: starting
s6-rc: info: service init-services successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun cron (no readiness notification)
services-up: info: copying legacy longrun nginx (no readiness notification)
services-up: info: copying legacy longrun php-fpm (no readiness notification)
s6-rc: info: service legacy-services successfully started
s6-rc: info: service 99-ci-service-check: starting
#2 {main}[ls.io-init] done.
s6-rc: info: service 99-ci-service-check successfully started
Your data directory is readable by other users.
Please change the permissions to 0770 so that the directory cannot be listed by other users.
Full addon config
Full_Text_Search: false
OCR: false
OCRLANG: eng
PGID: 1000
PUID: 1000
additional_apps: inotify-tools
certfile: fullchain.pem
data_directory: /mnt/Nextcloud
elasticsearch_server: ip:port
keyfile: privkey.pem
trusted_domains: redacted
use_own_certs: false
networkdisks: //<IP_ADDRESS>/i/Nextcloud
cifsusername: CoopNet Mainframe
cifspassword: redacted
System
Home Assistant 2022.9.1
Supervisor 2022.08.6
Operating System 8.5
Frontend 20220907.0 - latest
rpi 4

maybe related to issue #90

Hi, thanks very much for the report. Indeed when doing a local mount I put permissions as 755. I'll see if changing that helps

I've pushed a new version to test

🙏 you are a god among men, works perfectly.

oh, sidequest, this network share is on a windows computer. Is there anything I should do on that end to prevent any manipulation of files in the nextcloud dir?

uh did you modify anything related to trusted domains? it's now not letting me connect remotely. In my config I have "trusted_domains: domain.com, domain2.com" and it's not working now.

unfortunately, while it works, for some reason it's using the SD card on the pi still.

and my network share folder is empty even after uploading a few test files.
Log:
`To support LSIO projects visit:
https://www.linuxserver.io/donate/
GID/UID
User uid:    1000
User gid:    1000
cont-init: info: /etc/cont-init.d/10-adduser.sh exited 0
cont-init: info: running /etc/cont-init.d/20-config
cont-init: info: /etc/cont-init.d/20-config exited 0
cont-init: info: running /etc/cont-init.d/30-keygen.sh
[12:45:07] INFO: No ssl certificates set. Auto generating ones...
Generating a RSA private key
................................................................................+++++
........+++++
writing new private key to '/ssl/nextcloud/keys/cert.key'
... adding ssl certs in files
[12:45:07] INFO: ... done
cont-init: info: /etc/cont-init.d/30-keygen.sh exited 0
cont-init: info: running /etc/cont-init.d/40-config.sh
Creating folders
Updating permissions
cont-init: info: /etc/cont-init.d/40-config.sh exited 0
cont-init: info: running /etc/cont-init.d/50-install.sh
cont-init: info: /etc/cont-init.d/50-install.sh exited 0
cont-init: info: running /etc/cont-init.d/60-memcache.sh
[12:45:10] INFO: Setting data diretory: /share/nextcloud
cont-init: info: /etc/cont-init.d/60-memcache.sh exited 0
cont-init: info: running /etc/cont-init.d/90-custom-folders
cont-init: info: /etc/cont-init.d/90-custom-folders exited 0
cont-init: info: running /etc/cont-init.d/92-local_mounts.sh
cont-init: info: /etc/cont-init.d/92-local_mounts.sh exited 0
cont-init: info: running /etc/cont-init.d/92-smb_mounts.sh
Mounting smb share(s)...
Using PUID 1000 and PGID 1000
[12:45:12] INFO: ... //<IP_ADDRESS>/i/Nextcloud successfully mounted to /mnt/Nextcloud with options
cont-init: info: /etc/cont-init.d/92-smb_mounts.sh exited 0
cont-init: info: running /etc/cont-init.d/99-add_packages.sh
[12:45:12] INFO: Installing additional apps :
... inotify-tools
fetch http://dl-cdn.alpinelinux.org/alpine/v3.14/main/aarch64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.14/community/aarch64/APKINDEX.tar.gz
(1/1) Installing inotify-tools (<IP_ADDRESS>-r0)
Executing busybox-1.33.1-r8.trigger
OK: 289 MiB in 230 packages
cont-init: info: /etc/cont-init.d/99-add_packages.sh exited 0
cont-init: info: running /etc/cont-init.d/99-custom-files
[custom-init] no custom files found, skipping...
cont-init: info: /etc/cont-init.d/99-custom-files exited 0
cont-init: info: running /etc/cont-init.d/99-custom_scripts.sh
cont-init: info: /etc/cont-init.d/99-custom_scripts.sh exited 0
cont-init: info: running /etc/cont-init.d/99-elasticsearch.sh
Full_Text_Search option not set
cont-init: info: /etc/cont-init.d/99-elasticsearch.sh exited 0
cont-init: info: running /etc/cont-init.d/99-mariadb_discovery.sh
[12:45:17] WARNING: MariaDB addon was found! It can't be configured automatically due to the way Nextcloud works, but you can configure it manually when running the web UI for the first time using those values :
Database user : service
Database password : thoxeithaeCai6Mohquee6oNgahc1ceoRohJohshaiw9ToobaiT7aeyohxohy5ch
Database name : nextcloud
Host-name : core-mariadb:3306
cont-init: info: /etc/cont-init.d/99-mariadb_discovery.sh exited 0
cont-init: info: running /etc/cont-init.d/99-ocr.sh
cont-init: info: /etc/cont-init.d/99-ocr.sh exited 0
cont-init: info: running /etc/cont-init.d/99-trusted_domains.sh
[12:45:22] INFO: Currently set trusted domains :
<IP_ADDRESS>:8099
[12:45:24] INFO: Trusted domains set in the configuration. Refreshing domains.
[12:45:24] INFO: ... removing previously added trusted domain (except for first one created)
System config value trusted_domains => 2 deleted
System config value trusted_domains => 3 deleted
System config value trusted_domains => 4 deleted
System config value trusted_domains => 5 deleted
[12:45:31] INFO: ... alignement with trusted domains list : nc.obsidianmkt.com
[12:45:31] INFO: ... adding nc.obsidianmkt.com
System config value trusted_domains => 6 set to string nc.obsidianmkt.com
[12:45:32] INFO: Remaining configurated trusted domains :
[12:45:34] INFO: <IP_ADDRESS>:8099
nc.obsidianmkt.com
cont-init: info: /etc/cont-init.d/99-trusted_domains.sh exited 0
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service init-mods: starting
s6-rc: info: service init-mods successfully started
s6-rc: info: service init-mods-package-install: starting
s6-rc: info: service init-mods-package-install successfully started
s6-rc: info: service init-mods-end: starting
s6-rc: info: service init-mods-end successfully started
s6-rc: info: service init-services: starting
s6-rc: info: service init-services successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun cron (no readiness notification)
services-up: info: copying legacy longrun nginx (no readiness notification)
services-up: info: copying legacy longrun php-fpm (no readiness notification)
s6-rc: info: service legacy-services successfully started
s6-rc: info: service 99-ci-service-check: starting
[ls.io-init] done.
s6-rc: info: service 99-ci-service-check successfully started`
config:
Full_Text_Search: false OCR: false OCRLANG: eng PGID: 1000 PUID: 1000 additional_apps: inotify-tools certfile: fullchain.pem data_directory: /share/nextcloud elasticsearch_server: <ip:port> keyfile: privkey.pem trusted_domains: ********* use_own_certs: false networkdisks: //<IP_ADDRESS>/i/Nextcloud cifsusername: ***** cifspassword: *****

I navigated to /share/nexctcloud and the files are indeed there, but it didn't mount them to my network share, even though the log said it was mounted successfully.

Strange, I'll look at it tommorow. Thanks!

Actually I think I remember... The data directory is set at first run, where you can choose where it should be located, no? You could try making a backup, delete the next cloud  folder and addon, then reinstall to get the initiao setup form?
And this addon option only prefills the field, but has no effect after initial setup

Ok i'll give that a shot. I also noticed in the logs "[13:48:36] INFO: ... //<IP_ADDRESS>/i/nextcloud successfully mounted to /mnt/nextcloud with options "
it supposed to be /share/nextcloud not /mnt/nextcloud right?

ok no dice. I completely uninstalled nextcloud and mariadb, deleted the folder in /share/, reinstalled mariadb and then nextcloud, set everything up from scratch, same thing. files are on sd card in /share/nextcloud. the /nextcloud directory on my share is empty, files are on the sd card.

yeah I think that was the problem, I started over and changed the config to mnt/nextcloud from /share/nextcloud and all is well so far.

noooooooo so close. so now that it's populating files on my network share correctly, i'm getting "Error
Your data directory is readable by other users.
Please change the permissions to 0770 so that the directory cannot be listed by other users."
is it somehow detecting the share permissions on my smb share?

Ah, great, we're moving forward! I'll modify the documentation when I'll have more time to make more explicit that data folder is defined at first installation. For permissions, I'll push a new version now. Thanks very much for this troubleshooting!

Is this better?

i'm going to nuke the install and start from scratch. i'll be back.

unfortunately, same error.
"Error
Your data directory is readable by other users.
Please change the permissions to 0770 so that the directory cannot be listed by other users."

Well I thought I had disabled that from the options but it doesn't work. It's strange that suddenly it starts to enforces permissions, I'll look at it. Anyway you're data is safe, just slightly less accessible for the moment

well Im not worried about the data since I nuked it and started over, there's nothing there because I can't use nextcloud at all.

Have you set the data folder in the addon options to the actual remote folder?

not sure what you mean,
config:
Full_Text_Search: false
OCR: false
OCRLANG: eng
PGID: 1000
PUID: 1000
additional_apps: inotify-tools
certfile: fullchain.pem
data_directory: /mnt/nextcloud
elasticsearch_server: ip:port
keyfile: privkey.pem
trusted_domains: redacted
use_own_certs: false
networkdisks: //<IP_ADDRESS>/i/nextcloud
cifsusername: redacted
cifspassword: redacted

Could you please try data_directory: /mnt/Nextcloud? It is case sensitive and seems to be mounted with an uppercase in your log?

I noticed that a while ago and changed the share folder to lowercase to match. the logs are probably older when the share was uppercase.

Ah... Mounting the disk before creating the folder would help :) l'll update the code

i'll give it a shot. should I do a clean install, or can I just upgrade?

Honestly it solved an issue but still does not work... I'll continue tomorrow

ugh, well thanks for trying

Hi, as a workaround, there's a Nextcloud option to disable this check.
To do so, you have to enter the container of the addon (recommend Portainer for easy access).
Then edit the file /data/config/www/nextcloud/config/config.php. Here you'll find an option (and if there isn't, just create it) called check_data_directory_permissions and you have to make sure to set it to false ('check_data_directory_permissions' => false,).
Hope it helps!

Thanks, I've added it as an automatic step. I had put true... :-p

Ladies and Gentlemen.... WE GOT HIM.  Thanks for all your help over the last couple days. everything looks to be working perfectly so far

Yes, thanks very much for the feedbacks, troubleshootings, and solution ;-)


Introduce new catalog item type named Container

This work add new models and service models that allow to do provisioning through container templates, following the examples for Orchestration or AnsibleTower.
It also modified the controller and ui to allow to define a new catalog item type Container.
This request provides sufficient code to create a catalog item, together with #12598 ordering such service runs completed but does no actual container template deployment. The actual deployment logic is yet to be added in following up PRs.
No spec tests are added in the PR. They should be added together with the future PRs for container template deployment.

/cc @gmcculloug @simon3z

@zeari @zakiva can you review this? Thanks.

cc @lfu

@lfu I am closing this too, since UI has moved to its own repository. It can still be your reference when you start over the new implementation.


OCM-10607 | ci: fix the cluster leak issue

What this PR does / why we need it:
Cluster state file cannot be recorded once process terminated. This caused the resource leak in pull requests testing.
Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story (OCM-xxxx):
Fixes #
OCM-10607
Change type

[ ] New feature
[ ] Bug fix
[ ] Build
[x] CI
[ ] Documentation
[ ] Performance
[ ] Refactor
[ ] Style
[ ] Unit tests
[ ] Subsystem tests

Checklist

[ ] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.

/lgtm

/approve


Increase sidebar padding / top border

Is your feature request related to a problem? Please describe.
In the sidebar / menu the podium icon is obstructed by the header bar.
Describe the solution you'd like
Set padding so that the podium icon is shown in full.
Additional context

interested

Can I be assigned this

I would like to contribute to this issue!
I will increase the top padding and will make it beautiful.
can you please assign me !!

To avoid "issue squatting" during Hacktoberfest, issues are assigned retrospectively following a PR for first time contributors.


fix(documentation):  Switch from css to scss highlighting in .mdx.

Only the highlighting in checkbox.docs.mdx needed to be changed. All the other things had already been done

I propose to close this PR in favor of https://github.com/swisspost/design-system/pull/1928

As the PR #1928  will overhaul all the scss imports this changes are obsolete


Add delay for ack packets

Currently, renet sends acks every update frame, this increases the bandwidth for connections.
This PR adds a delay for sending ACK packets, when receiving a reliable message this delay is ignored and the ACK is sent immediatly (so we don't delay acks for reliable messages).
This reduces the bandwidth of connections. For idle connections this seems to reduce from 2.15 kbps to 0.5 kbps (at 60 hz and 200ms max delay). If the connection is receiving reliable messages every frame this PR will not change the bandwidth since it will still send ACKs every frame.
This change will affect a bit the stats for the connection , since the values can be delaye, but at small MAX_ACK_DELAY ( 200-300ms), this shouldn't impact much.
TODO: make sure the RTT calculations is working correctly and check that the new delay field in the ACK packet cannot be wrongly used with malicious packets.

Why put the ack delay in the ack packet, and not as a config? This makes ack packets bigger and adds risk of malicious effects. It also breaks from the netcode standard.

Just a late reply, this changes the custom renet protocol, shouldn't interfere with the netcode protocol.
The delay is added to the ack packet so that we can use it to calculate the RTT, if the ack delay is 0, and we know that the ack was sent instantly. Otherwise, if the packet had some delay, we sent how long it waited, so when it is received it can be used to calculate the correct RTT.
So why do this at all. If we don't send any reliable messages, we would never send acks instantly and without the delay specified in the ack packet, we could not calculate the RTT correctly.

What about adding the config to ConnectionConfig? Users are already expected to use the same channels, so including ack delay wouldn't be too unusual. You can make it Option<Duration> and then default to 60Hz if not set (or use Some(60Hz) as the default when constructing config, and allow acks to be sent every tick when None).

What about adding the config to ConnectionConfig? Users are already expected to use the same channels, so including ack delay wouldn't be too unusual. You can make it Option<Duration> and then default to 60Hz if not set (or use Some(60Hz) as the default when constructing config, and allow acks to be sent every tick when None).

In my mind, this is not something a user should need to configure or know/worry about. Most of the times this value won't impact much because if any reliable packet is received it will always send an ack immediatly. A value between 250 - 500ms should be good enough. Also, this would not be an intuitive config for some users.

How about instead of Duration you store a u16 for the number of milliseconds delayed?


Fix bad auto deduction variable type in JPetUnpacker

Change auto to std::string to force std::string type after calling JPetCommonTools::stripFileNameSuffix.
Automatic variable type deduced const char *, but it was passed to std::string& and in the log you could see, that unzipped name was not correct:
72: [JPetUnpacker.cpp:exec@43] ThreadID: 0x00007fdaa221ab00  The hld file doesnt exist: \C0\861\B7\ECU
Also add flag to return status of unpacking to exit program if unpacking was not successful.
Beside of that, I removed inheritance of TObject from JPetUnpacker, it produced warning:
Warning in : The data members of JPetUnpacker will not be stored, because it inherits from TObject but does not have its own ClassDef.
and I couldn't find anywhere where something from TObject was used in JPetUnpacker

I refactored full run method, I think now It is much more clear. I left fUnpackHapped variable, because it is used in terminate to change some userParams and added new local variable to run.
Local variable runStatus tells if there was some error during unzipping/unpacking, but it is also true, when input file was root or other type that do not require any unzipping/unpacking.
If input file type will be undefined I'm returning false and terminating analysis.

Very nice!!!


Shortcut for including external libraries / functions

Would be nice if ng-webworker provided a shortcut for including external libraries (by using importScripts). I can think of several libraries that could be used in a computation thread, such as numeric.js and lodash.js. Some WW abstraction libraries even allow to pass arbitrary functions.

Sure, you can use importScripts. Your code runs inside a web worker. So you can add any native web worker functions. I just added a demo for you at the bottom of my demo site.
http://mattslocum.github.io/ng-webworker/demo/


Action Purge doesn't actually purge packages

If a package is removed, but not purged and you use the action purge nothing happens:
Example:
package %w(popularity-contest whoopsie landscape-common command-not-found command-not-found-data) do
action :purge
end
From the Chef run:
Recipe: core::packages

apt_package[popularity-contest, whoopsie, landscape-common, command-not-found, command-not-found-data] action purge (up to date)

DPKG output:
dpkg -l | grep landscape
rc  landscape-client                 14.12-0ubuntu0.14.04                amd64        The Landscape administration system client
rc  landscape-common                 14.12-0ubuntu0.14.04                amd64        The Landscape administration system client - Common files

So, this is going to be kind of invasive to the existing provider code.  Right now you're either an installed_version or not.  The logic is boolean, not trinary.  Adding another state for removed-but-not-purged is going to be complicated and messy, particularly after the multipackage stuff.
The provider really /should/ work this way, but given the effort involved vs. the payoff, the priority of fixing this is pretty low.
Also, the apt provider is already a horrible mess and really needs cleanup before we start adding more features to it.  That and the yum-provider are my go-to examples of how accreting more and more open source PRs for features is not good software development.


[REQ] Drag and drop icons to buttons

I am migrating my StreamDeck configuration to unix-streamdeck, but clicking Select Icon and navigating to the required image (even with a bookmark in place) is a slow process.
Please add the ability to drag and drop icons on buttons to speed up the process of laying out and arranging the icons.
Thanks for making this project, it is shaping up to be something quite special 💖

I'm looking into this, although it may take some time, because the upstream framework we use for the UI doesn't currently support this.

Thanks for looking into this :slightly_smiling_face:
For anyone following this bug, here's a workaround of sorts:

Open the buttons icons you are want to add to StreamDeck Unix in an image viewer, this places the icon in the zenity Recent list and makes bulk-adding icons much quicker.

Interesting workaround.
Using the last opened folder would also be a big improvement.


How to use with webpack / react

Hi! I'm seeing Uncaught TypeError: fs.readFileSync is not a function. Any suggestions? Have tried importing from dist in addition to the normal package.

@privateOmega is https://github.com/privateOmega/html-to-docx/pull/68 intended to address this?

Duplicate of https://github.com/privateOmega/html-to-docx/issues/3

@amilich How have you installed the package? try the steps mentioned towards the end in #3

@amilich How have you installed the package? try the steps mentioned towards the end in #3

I've tried, I now see Internal Error: Invalid descriptor (https://github.com/privateOmega/html-to-docx.git#feature/browser-example) on yarn2

@amilich yarn add <package name>@git+ssh://<git remote URL>#semver:^3.2.0 this is the format for yarn2, not yarn add url. You have to explicitly specify package name I guess.
Lifted from here https://github.com/yarnpkg/berry/issues/994#issuecomment-628586986

@amilich yarn add <package name>@git+ssh://<git remote URL>#semver:^3.2.0 this is the format for yarn2, not yarn add url. You have to explicitly specify package name I guess.
Lifted from here yarnpkg/berry#994 (comment)

yarn add html-to-docx@git://github.com/privateOmega/html-to-docx.git#feature/browser-example seems to have worked!

@amilich yarn add <package name>@git+ssh://<git remote URL>#semver:^3.2.0 this is the format for yarn2, not yarn add url. You have to explicitly specify package name I guess.
Lifted from here yarnpkg/berry#994 (comment)

yarn add html-to-docx@git://github.com/privateOmega/html-to-docx.git#feature/browser-example seems to have worked!

Still seeing package unable to be resolved... not sure what's up. Have had issues adding via GH urls before

@amilich yarn add <package name>@git+ssh://<git remote URL>#semver:^3.2.0 this is the format for yarn2, not yarn add url. You have to explicitly specify package name I guess.
Lifted from here yarnpkg/berry#994 (comment)

Do you plan on merging https://github.com/privateOmega/html-to-docx/pull/68 soon?

@amilich Please test with latest version 1.2.1.


luci-app-nextdns: add luci integration for nextdns package

Signed-off-by: Olivier Poitrey<EMAIL_ADDRESS>Tested on 19.07.0-rc1

@hnyman here it is


/node_modules/ng2-uploader/ng2-uploader.js  not found

After upgrading to 0.5, I get the error that ng2-uploader can not be found. Looking under the node_modules folder, looks like the ng2-uploader.js file is not being generated from the .ts file. This was not the case with earlier versions.
Folder structure with version 0.5.0 after npm install:
ng2-uploader
|--node_modules
|--src
|--ng2-uploader.ts
|--package.json
|--README.md
|--tsconfig.json
Worked fine with 0.3.5. File structure with version 0.3.5
ng2-uploader
|--bundles
|--config
|--src
|--tests
|--ng2-uploader.d.ts
|--ng2-uloader.js
|--ng2-uploader.js.map
|--ng2-uploader.ts
|--package.json
|--README.md
|--tsconfig.json

Hi, I have the same error
and this Warning:
"file was not compiled because there is no a reference from tsconfig.json"
related to ng2-uploader.ts

+1. @jkuri  This package went from usable and working immediately with instructions how to load it through systemjs to.... no compiled js included and no docs on how to load it. What happened?

Sorry guys, I want it to prepare to work with angular-cli install command, and the source .ts files are all that will be needed.
It's still usable, just copy the whole directory to somewhere in the project you have source files and then use the correct path to ng2-uploader.ts when importing UPLOAD_DIRECTIVES.

@jkuri Could you please upload a 0.5.1 package to npm that includes the .js (+ .js.map) files that come out of the .ts files? So we don't have to transpile ourselves. That is what all library developers do. And then they include a bundled .js file too (although I don't really need it).

Hi @mwijnands, okay, I will do that in the near future and let you know.

@mwijnands published 0.5.1 as per you request.

Thanks! :smile:


[FLINK-8500] Get the timestamp of the Kafka message from kafka consumer

What is the purpose of the change
This pull request make the Kafka timestamp and timestampType available in the message deserialisation so one can use it in the business logic processing.
Brief change log
Introduced new interface ConsumerRecordMetaInfo with meta info of the kafka message
Extended the DeserializationSchema with the T deserialize(ConsumerRecordMetaInfo consumerRecord) method.
Adjusted the Kafka Connectors to support the new interface.
Added some documentation.
Verifying this change
This change is already covered by existing tests, such as most of the Kafka Consumer tests.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Thanks for the update @FredTing.
I'll try to take another look at the PR within the next days.

There are a few bigger design aspects that we need to agree upon:

The DeserializationSchema is a shared common denominator of serialization schemata. That's why it is in flink-core and not in a Kafka connector project. It is used by various per-record streaming sources, like Kafka, RabbitMQ, in the future PubSub, or AMQ. It may be valuable to migrate Kinesis also to that. This PR changes the common denominator to have very Kafka-specific fields.

The common schema makes sense because we can offer a library of implementations, like for Avro, Json, Thrift, Protobuf. All connectors can use hardened implementations for these types, or integrations with schema registries.

This surfaces for example in the SQL / Table API, which is currently making an effort to change their source API to have "connector" and "format" aspects orthogonal. You define a table as "from kafka with Avro", or "from row-wise file with JSON", etc.

We should think of having something similar in the future in the unified batch/streaming DataStream API as well, when we rework our source interface. At least a "row-wise source" that can then use all these format implementations.

That means we are in a bit of a conflict between "common deserialization schema" interface and surfacing connector specific information.
One way to approach that might be making the connector-specific deserializer classes subclasses of the common one, and let them use specialized subclasses of ConsumerRecordMetaInfo that have the additional fields.
On a separate note, I think that ConsumerRecordMetaInfo is not the best name, because the type has not only the meta info, but the actual record. So we could call it Record or Datum or SourceRecord, etc.

If it turns out that we need to do a bit more design work on the deserialization schema, we could incrementally fix the issue that triggered this PR by actually extending the KeyedDeserializationSchema with a new default method.

@FredTing we had some offline discussion on how to proceed with this.
@aljoscha, @twalthr, or @StephanEwen can probably comment more here if I missed anything.
The conflict that Stephan mentioned between a "common deserialization schema" interface and exposing surfacing connector specific information is rooted in the fact that both concerns (deserialization and providing connector specific record meta information) is currently coupled in a single interface.
Take for example the Kafka connector's KeyedDeserializationSchema - there we try to deserialize the Kafka bytes, as well as provide information such as topic / partition / timestamp etc. to allow the user to enrich their user records for downstream business logic. The first part (deserialization of bytes) should be something common for all connector sources, while the second part is Kafka-specific.
Therefore, we should perhaps break this up into two separate interfaces, as follows:
// common interface for all sources (we already have this)
interface DeserializationSchema<T> {
    T deserialize(byte[] bytes);
}

// ... and a Kafka-specific interface that is only used to provide record meta information
interface ConsumerRecordMetaInfoProvider<T> {
    T enrich(T record, ConsumerRecordMetaInfo metaInfo);
}

The second interface is something that each connector should have independently, and does not handle deserialization of the record bytes. The name, of course, is still open to discussion.
What do you think?

Closing this PR because new insights