Support for rate limiting

Thank you for making this!
I wonder if you have an example of how to stay within the Shopify rate limits given here:
https://shopify.dev/docs/api/usage/rate-limits
I know there are crates that allows setting up client rate limits like this (Governor, reqwest-ratelimit), but it isn't entirely clear to me how this would be done in this case. But I wonder if there was maybe room for a feature in the crate itsealf that would enable something to take care of this. Like a feature called governor which would bring in automatic rate limitation. I am not a super advanced rust user so I don't know how concerns are normally split in this ecosystem and if this is something end users are expected to take care of themselves. But having an example of how to do it would be nice as I imagine it is something many users could be interested in.

Just to throw my 2 cents in (I am not a galaxy brain Rust user) - I think it should really be up to the end user on how they want to implement rate limiting rather than have the 'magic' inside the crate itself.
For staying within the limits - in my experience running a 10k+ SKU (hardly anything compared to some) Shopify site - revert everything you can to bulk actions. This will make the rate limits redundant where you can default to these anyway. If you haven't touched Shopify in a while they recently upgraded their product/item data model which gives some better features (more bulk action availability) - https://shopify.dev/docs/apps/build/graphql/migrate/new-product-model.
If you can't use bulk queries (there is still heaps of areas lacking) I am 90% certain the graphQL endpoint returns an X-remaining-limit header which you can read and fallback appropriately - you'll just need to pass an empty response_finder. This is how you could implement something which would be dirty and nasty.
let set_query_response_finder = vec![]; // blank is fine. This is required by the lib.

  let set_inventory_mutation: Result<GraphQLResponse, ShopifyAPIError> = shopify_client
    .graphql_query(query, &set_mutation, &set_query_response_finder)
    .await;

  match set_inventory_mutation {
    Ok(response) => {
      let throttle_status = response.extensions.cost.throttleStatus;

      if throttle_status.currentlyAvailable < 100.00 {
        // delay
        tokio::time::sleep(std::time::Duration::from_secs(20)).await;
      }

      Ok(())
    }
    Err(e) => {
      // logging
      tracing::error!("Error setting inventory: {}", e);

      Ok(())
    }
  }

Hope this helps!

Thats a good consideration. But I think you can only have one bulk query executing at a time though right? I have several different endpoints that would need to do this, and I am sure there would be overlap in when they are called so then I would need to handle an extra state of reporting back to the user than I am not ready to
I ended up going a bit of a different route and not using this library in the end, but I solved my requirement of handling rate limitting by using the leaky-bucket crate. And then I check the cost of every query and when I call my client it calculates if the request might get rate limited. And as I go along I keep updating the current rate limiting status based of what Shopify admin returns.
I am not sure if the same could be done with the library, but here is a snippet of what I did in the end:

#[derive(Debug, Clone)]
pub struct Client {
    url: Url,
    client: reqwest_middleware::ClientWithMiddleware,
    rate_limiter: Arc<Mutex<RateLimiter>>,
}

/** We never want to get too close to the Shopify API rate limit, so we add a bit to our estimated cost */
const BUFFER_COST: usize = 40;

impl Client {
    /**
    Run a query against the Shopify GraphQL API.

    This function will automatically handle rate limiting.
    */
    pub async fn send<T, U>(
        &self,
        body: QueryBody<T>,
        estimated_query_cost: usize,
    ) -> Result<U, Error>
    where
        T: serde::Serialize + Debug,
        U: serde::de::DeserializeOwned,
    {
        let safe_cost = estimated_query_cost + BUFFER_COST;
        {
            let rate_limiter = self.rate_limiter.lock().await;
            let could_acquire = rate_limiter.try_acquire(safe_cost);
            if !could_acquire {
                tracing::trace!(
                    "Expected to be rate limited with query estimated cost {}, balance is {}",
                    safe_cost,
                    rate_limiter.balance()
                );
                rate_limiter.acquire(safe_cost).await;
            }
        }

        tracing::trace!("Making request to Shopify GraphQL API");
        let res = self
            .client
            .post(self.url.clone())
            .json(&body)
            .send()
            .await?;
        tracing::trace!("Parsing response from Shopify GraphQL API");
        let response_body: graphql_client::Response<U> = res.json().await?;

        let extensions = Extensions::try_from(response_body.extensions.clone());
        match extensions {
            Ok(extensions) => {
                let throttle_status = extensions.cost.throttle_status;
                let mut rate_limiter = self.rate_limiter.lock().await;
                *rate_limiter = build_rate_limiter(
                    throttle_status.maximum_available,
                    throttle_status.currently_available,
                    throttle_status.restore_rate,
                );
            }
            Err(err) => {
                tracing::warn!("Failed to parse extensions: {}", err);
            }
        }

        if let Some(errors) = response_body.errors {
            let errors = errors
                .iter()
                .map(|error| error.message.clone())
                .collect::<Vec<String>>()
                .join(", ");
            return Err(Error::Response(errors));
        }

        response_body.data.ok_or(Error::MissingData)
    }
}

pub fn build_client(
    shop_name: String,
    shopify_api_token: String,
) -> Result<Client, BuildClientError> {
    tracing::info!("Building reqwest client");
    let mut default_headers = reqwest::header::HeaderMap::new();
    default_headers.append(
        "X-Shopify-Access-Token",
        reqwest::header::HeaderValue::from_str(shopify_api_token.as_str())?,
    );

    let client = reqwest::Client::builder()
        .default_headers(default_headers)
        .build()?;

    let client = ClientBuilder::new(client)
        .with(TracingMiddleware::default())
        .build();

    const ASSUMED_MAX: usize = 2000;
    const ASSUMED_INITIAL: usize = ASSUMED_MAX;
    const ASSUMED_REFILL: usize = 100;
    let rate_limiter = build_rate_limiter(ASSUMED_MAX, ASSUMED_INITIAL, ASSUMED_REFILL);

    const SHOPIFY_API_VERSION: &str = "2024-07";
    let url = Url::parse(&format!(
        "https://{}.myshopify.com/admin/api/{}/graphql.json",
        shop_name, SHOPIFY_API_VERSION
    ))?;

    Ok(Client {
        url,
        client,
        rate_limiter: Arc::new(Mutex::new(rate_limiter)),
    })
}

/** The rate at which the rate limiter refills is always per second. */
pub static RATE: Lazy<Duration> = Lazy::new(|| Duration::from_secs(1));

/** Build a leaky bucket rate limiter with the given parameters. */
pub fn build_rate_limiter(max: usize, intial: usize, refill: usize) -> leaky_bucket::RateLimiter {
    leaky_bucket::RateLimiter::builder()
        .max(max)
        .initial(intial)
        .refill(refill)
        .interval(*RATE)
        .build()
}

I had a usecase of just running a bunch of mutations in paralell and with this they seem to wait for the balance to build up and avoid getting rate limited by the api server. Again, I am new to Rust so its entirely possible that some part of this is way I have done it is a terrible idea, but for my immediate testing it seemed to work nicely.


Renaming of GO terms GO:0031771 and GO:0031772

Dear GO team
In relation to a UniProt user request to update the naming of UniProt entries O43612 (human HCRT) and P01189 (human POMC), the user suggested the renaming of the GO terms GO:0031771 "type 1 hypocretin receptor binding" and GO:0031772 "type 2 hypocretin receptor binding". The user suggested to use the names "OX1 and OX2 orexin receptors"  (the digits in subscript) and change the GO terms to "OX1 orexin receptor binding" and "OX2 orexin receptor binding", respectively.
Would you be so kind to look into this matter and let me know of your decision, so I can forward your reply to the respective user?
Thank you very much for your time.
Best wishes,
Yvonne

Hi Pascale
Thank you very much for your reply about this issue. I understand you are avoiding acronyms for GO terms. Your suggestion to replace 'hypocretin' by 'orexin' and using 'OX1/OX2'  in the synonyms sounds like a good solution to me.
Thank you very much and best wishes, Yvonne

Also changed
GO:0061585
-name: hypocretin secretion, neurotransmission
+name: orexin secretion, neurotransmission
GO:0061584
-name: hypocretin secretion
+name: orexin secretion
Thanks, Pascale


Master/mongo connectivity issues

When doing a MITG style deployment:
root/initial VMs: initial-master-node, mongo-node

initial-master-node cloud config starts a master that points to mongo-vm (that runs mongodb with cloud config)
initial-grid is created here

master-grid VMs: mitg-node-0, mitg-node-1, mitg-node-2

these nodes run kontena-agent and they join the initial-grid
mongo replica set is deployed as a kontena service (daemon) on these nodes (as mongo-peer-1, mongo-peer-2, mongo-peer-3)
kontena master is deployed as kontena service (daemon) on these nodes (as master-1, master-2, master-3) and mongo is configured as MONGO_URI=mongo://mongo-peer-1:27017,mongo-peer-2:27017,mongo-peer-3:27017

Problem:
t0: mongo replica set is started
t1: masters are started, ruby(?) resolves mongo-peer-1..3 as <IP_ADDRESS>..103
t2: mongo replica set restarts, mongo-peer-1 is now <IP_ADDRESS>..203
t3: masters have stale DNS and are thus unable to connect to the mongo nodes. master constantly throws exceptions on unable to connect to the replica set, but does not die.
t4: operator manually restarts masters, DNS is refreshed --> works again.
t5: GOTO t2
Suggested actions:

Somehow force refresh of DNS records..?
If the master is unable to connect to mongo --> exit and docker will restart

prob. due to https://github.com/mongoid/moped/issues/236

I guess the real problem is that master is considered healthy in this state

https://github.com/kontena/kontena/pull/1290

https://github.com/kontena/kontena/pull/1580 sort of fixes this I thinks

#1290 should fix this properly

#2257 should fix this (again)


Extreme memory usage during JXL decode (GraphicsMagick oss-fuzz 69728 test case #2)

Describe the bug
Libjxl consumes huge amounts of heap memory ( 3,393,388,337 bytes) while decoding this tiny (127 bytes!) file
With a somewhat older libxl, there was also this unusual exception:
zsh: illegal hardware instruction (core dumped)

To Reproduce
djxl<PHONE_NUMBER>813184-tc2.jxl crap.pnm

With this test case:

Expected behavior
Libjxl should detect an irrational request and reject decoding prior to allocating huge amounts of memory.
 Not sure what you expected here.
The command-line frontends will just fail the allocation and there's no harm. If you're linking to libjxl yourself, you can always just check the image dimensions before you attempt to allocate an output buffer.


no-unused-variable: shouldn't warn about all params in some cases

Bug Report

TSLint version: 3.11.0
TypeScript version: 1.8.10
Running TSLint via: gulp-tslint via Visual Studio 2015

TypeScript code being linted
this.$scope.$on("$myEvent", (event: angular.IAngularEvent, customerNumber: string) => {
    this.getCustomerInfo(customerNumber);
});

with tslint.json configuration:
"no-unused-variable": [ true, "check-parameters" ],

Actual behavior
The event parameter is reported as unused, which is technically correct... but it has to be there so I can access the 2nd parameter, which I need.
Right now I'm needing to put comments like // tslint:disable-next-line:no-unused-variable all over the place.
Expected behavior
Since I only need to use the 2nd parameter, it seems dumb to warn me about the first parameter not being used since this is how the framework I'm using works.

You can use the ignore-pattern option in this case to specify a regex that corresponds to variables that are intentionally unused. For example,
"no-unused-variable": [true, {"ignore-pattern": "^_"}]

would ignore all variables that start with an underscore and not report errors when they are unused.

so I'd have to go through my code and rename variables based on their parameter order just to satisfy this rule?

Yep, but it's not that unusual: starting unused variables with an underscore is a standard convention in languages with stricter compilers, like Go and Rust, as far as I know.

So if I were to rename that first var to _event and later on it was used by someone who was not aware of this convention, would that be flagged?

Good question - I believe the rule only works one way here. TSLint won't complain if you use a variable that meets the "ignore-pattern" option

Perhaps that could be a configurable option to add to this rule?

to me it is a bug, tslint should not throw error in this case.

@marcoturi no-unused-variable is deprecated in favor of compiler options and we plan to remove it in the next major version, see #1481. I filed #1851 to address the sparse documentation around this.


Support for Portable Air Conditioner DR-HAC005S

Hello, according to this page it only supports Tower Fans, Air Circulators and Space Heaters at this time, but I figured I would put in a request for support for Portable Air Conditioner DR-HAC005S.  Thanks!

I confirmed this actually worked even though the page doesn't mention Portable Air Conditioners.  Closing this ticket.  Thank you for your efforts!


Feature/sphinx version fix

Still using an older version, but it doesn't have problems due to its unpinned requirements.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (b806e05) 74.28% compared to head (e702783) 74.28%.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #148   +/-   ##
========================================
  Coverage    74.28%   74.28%
========================================
  Files           10       10
  Lines         3691     3691
========================================
  Hits          2742     2742
  Misses         949      949

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Create a logo

The site is really minimalist both by design (it's fast!) and need (I wanted to get something out the door while focusing on writing). That said, I think there's value in having an Ethical Web Development logo. If nothing else I'd love to have hex stickers to pass out at conferences, etc.
Here's my first attempt:

Thoughts? I'd sincerely appreciate feedback.

As soon as I posted this I realized how similar it is to the MIT logo. Darn subconscious.

This looks kind of...impersonal to me. It seems like the logo should invoke the feeling of ethics, and ethical behavior. Maybe incorporate human hands, two humans facing each other, etc.?

You're completely right about that @danmunz. Thank you!

I think the Ruby For Good logo does a good job of balancing a clean aesthetic with a welcoming design:

(I'm so not good at branding)

In lieu of the A.I. storm, have you considered trying to have an AI generate a logo? If nothing else, it'll give you a few good laughs :wink:


Add a GitHub activity tracker

We could add a small section on the site to mention the following:

[ ] Latest merged pull request
[ ] Live pull requests

Not a major use case, May have to close


Wrong translation values for 'Member Collectives' titles

It looks like the translation values of collective.collective.memberOf.collective.parent.title is set to the template for the subtitle for non-English languages, and not the actual translation of "Member collectives".
See the English translation vs French translation.

@hichamelkaddioui you're absolutely right. I prepared a PR with that change.
https://github.com/opencollective/frontend/pull/646
Do you see anything else worth fixing?

@znarf French translation also differs:

on line 204 (en)
on line 547 (en)

I'm not sure those are really bugs, but it may be desirable translations carry the same message?

@hichamelkaddioui thank you, fixed these too in the PR.


Improve shortcuts view

Looking for shortcuts is a recurring issue.
We need a more expressive shortcuts view to support that.

Made the view show all shortcuts defined locally and those defined in the parents. The element column only shows the elements which are different than the current one.


add wipe flag for ctx and session

What this Pull Request (PR) does
A wipe parameter is added to context and session for cleaning up outdated data
Screen

Hi, thank you for the PR.
You don't need to implement the Wipe methods in Sessions and Contexts, the Delete method is already available inside of those structs by Storage struct, just call it.

@eugeis Thanks for the guidance, removed the Wipe method and called the Delete method of the Storage directly.


feat(upgrade): add handlers for conversion of CRD's in OSM

Description:
This commit adds a handler for all the CRD's that OSM manages. The
actual business logic for conversion will be added once a CRD has
multiple versions.
Fixes #3396
Signed-off-by: Sneha Chhabria<EMAIL_ADDRESS>Affected area:

Functional Area

New Functionality
[ ]

Documentation
[ ]

Install
[ ]

CLI Tool
[ ]

Certificate Management
[ ]

Control Plane
[ ]

Ingress
[ ]

Egress
[ ]

Networking
[ ]

Observability
[ ]

SMI Policy
[ ]

Sidecar Injection
[ ]

Security
[ ]

Upgrade
[X]

Tests
[ ]

CI System
[ ]

Demo
[ ]

Performance
[ ]

Other
[ ]

Please answer the following questions with yes/no. no

Does this change contain code from or inspired by another project? no

Did you notify the maintainers and provide attribution?

Is this a breaking change? no

Codecov Report

Merging #3800 (d652b0b) into main (80fa5ad) will decrease coverage by 0.94%.
The diff coverage is 5.03%.

:exclamation: Current head d652b0b differs from pull request most recent head 90d10de. Consider uploading reports for the commit 90d10de to get more accurate results

@@            Coverage Diff             @@
##             main    #3800      +/-   ##
==========================================
- Coverage   68.48%   67.54%   -0.95%
==========================================
  Files         189      198       +9
  Lines       10769    10928     +159
==========================================
+ Hits         7375     7381       +6
- Misses       3345     3498     +153
  Partials       49       49

Flag
Coverage Δ

unittests
67.54% <5.03%> (-0.95%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/crdconversion/egresspolicyconversion.go
0.00% <0.00%> (ø)

pkg/crdconversion/framework.go
0.00% <0.00%> (ø)

pkg/crdconversion/httproutegroupconversion.go
0.00% <0.00%> (ø)

...g/crdconversion/ingressbackendspolicyconversion.go
0.00% <0.00%> (ø)

pkg/crdconversion/meshconfigconversion.go
0.00% <0.00%> (ø)

pkg/crdconversion/multiclusterserviceconversion.go
0.00% <0.00%> (ø)

pkg/crdconversion/tcproutesconversion.go
0.00% <0.00%> (ø)

pkg/crdconversion/trafficaccessconversion.go
0.00% <0.00%> (ø)

pkg/crdconversion/trafficsplitconversion.go
0.00% <0.00%> (ø)

pkg/crdconversion/crdconversion.go
66.30% <100.00%> (+2.01%)
:arrow_up:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 80fa5ad...90d10de. Read the comment docs.


DAFFODIL-2169 Add typesafty to our handling of infoset data values.

For performance reasons, we have been making extensive use of
AnyRef to store data values in our infoset, and throughout the runtime.
This allows us to avoid boxing, but prevents the type system from validating
that unexepected types to not infect our values.
This commit introduces a value type, DataValue, which provides a 0 overhead (eg. no boxing)
wrapper around AnyRef, while only allowing a limited number of types to populate it.
In doing so, an unrelated bug of mixing Java and Scala BigInts was uncovered and corrected.
Such a bug should be much less likely to occur in the future.
DAFFODIL-2169
DAFFODIL-2188 (unconfirmed)

There are no new tests under daffodil-test in this squashed commit.
Is there a TDML test that exercises the enum feature the way we expect customers will use it?

There are no new tests under daffodil-test in this squashed commit.
Is there a TDML test that exercises the enum feature the way we expect customers will use it?

There is no functional change in this commit.
I did fix to bugs/mistakes, however:

1 was simply dead code. We were testing if the result of eval was ever a PState.
A second was a Scala BigInt vs Java BigInteger mistake. This turned out not to be the cause of the type-calc issue, and I do not know of a code path that would expose this mistake (not that I looked particuarly hard for one).

We have some tests from the original PR at daffodil-test/src/test/resources/org/apache/daffodil/extensions/type_calc
None of these were created with pedagogy in mind, but some of the early examples  in inputTypeCalc.tdml (namely the keysetValue examples) show the simple usecase; where the parser consumes a byte and outputs a string and the unparser does the inverse.
I am working on a seperate PR to fix the original typeCalc problem that started this. That fix builds off of the work here, but involves a fairly significant refactoring of the typeCalc system, so I want to keep it as a seperate PR.
That PR will add a TDML test that demonstrates the more advanced usecase where both the integer and string representations are preservered in the infoset.
For pedagogical purposes, it might makes sense to create a separate tdml file with just example test cases. As is, the useful examples are mixed in with more unit-testy ones.


🧑⚕ Artefacts for the Analytical Platform ITHC scoping doc

We are having a ITHC soon and need to update the diagram and docs for this work to proceed.
Definition of Done

[x] Updated HLD
[x] Update scoping document for Simon

Updated scoping document


Snapshot fetch: support resumes and streaming

Issue summary
Thanks to the herculean work with TypeScript performed in the latest snapshot worker, we are now ready to support resumes for fetching the snapshots from CloudFlare.
We also moved away from aria2c so we need to implement this in terms of reqwest (or some other solution that doesn't require external binaries). One library, being abandonware at the moment, we can draw inspiration from is reqwest_resume. It should do what we want, but it is no longer maintained, and we don't want to maintain a fork. So only draw inspiration from it.
If feasible, it would be great to have a streaming solution for this so the result could be piped to other commands. If it proves challenging, let's create a separate issue.
Tasks:

[ ] Implement resumes. Either use the library provided if it works or contrive our own solution if necessary.
[ ] Unit tests for resumes.
[ ] Implement streaming for fetching or create an issue if it requires more work than anticipated.
[ ] Unit tests for streaming if implemented.

Acceptance criteria:

[ ] Resuming works and a set of unit tests is covering the feature,
[ ] Streaming is implemented (and unit-test covered) or an issue is created.

Other information and links

I might not have time for this soon; bailing for now. If anyone is interested, go ahead and grab it.

Resume for streams implemented. Resuming partially downloaded files has yet to be implemented, but it is a lower priority.

Let's skip resume for downloading files. Users can always use an external tool if need be.


Added snowflake_query_history model, disabled by default

I was unable to test this alongside the rest of the package as the macros were failing in master; on removing the macros it compiles and runs correctly.
Can take a while for first run; set max_load_minutes to 60 for a short first run.

Thanks for the contribution, @ciejer! I'll review and test the code.
What was the error message you were getting with the macros in?

Oh I know. 🤦 Missing a raw tag in the docs file.


App Stopped working

When i call fragmenttransaction.commit() is gives me fatal exception main error so can anyone help me out with this ,,,,??

Hello, @HasnainKhanNiazi. We couldn't reproduce this issue. The best way for you to check various approaches of using this lib is to look inside paper-onboarding-fragment-example and paper-onboarding-simple-example modules. Everything works perfectly there.


dwa_algorithm

Adding machine learning (ML) to improve the performance of the robot's routing and scheduling via using the information collected (such as path length, optimised path, computation time, etc.) to train a model that can predict better paths or tune the parameters for optimization. The basis implementation approach to incorporate ML are shortest path, least computation time.
State: The robot's current position, distance to the goal, distance to obstacles, etc.
Action: Possible movements (e.g., velocity, yaw rate).
Reward: A function that rewards shorter paths, lower computation time, avoiding obstacles, etc.
We used the algorithm - Deep Q-Networks (DQN) to create an environment class that encapsulates the state, actions, and rewards. This environment interacts with your RRT and DWA algorithms.
The DQN will be responsible for learning the optimal policy based on the robot's state (current position, distance to the goal, distance to obstacles, etc.), actions (possible movements), and rewards (based on path length, computation time, avoiding obstacles, etc.).
Distance to the goal: Encourage the drone to get closer to the goal.
Obstacle clearance: Penalize the drone for getting too close to obstacles.
Path smoothness: Encourage smoother paths by penalizing sharp turns.
Energy usage: Penalize higher energy consumption to encourage efficient paths.


Add local project file (pwd/.tmuxinator.yml) support.

We have a need to be able to commit project files to git and share them
between machines (we use disposbale development boxes).  Refactored
Tmuxinator::Config.project to also check for $PWD/.tmuxinator.yml
when searching for project files.  This occurs between the recursive
search of Tmuxinator::Config.root and the default path being selected,
so as to allow for project files to be shared with no disruption to
existing functionality.

+1. The alternative workaround requires us to add our .yml files to the root via configuration managment, instead of within the project's code where it belongs.

It would be great if this could handle not having to pass in the name of the project when the file is present. So just running mux should start the project. This would require modification to bin/tmuxinator though.
Lastly when doing a mux new we should probably add an option (--local) to create the file in the current directory instead of ~/.tmuxinator.

@Soliah added requested mux new --local and no-arg mux to create a local project and start a local project, respectively.  Didn't see any tests to handle modifications to bin/tmuxinator; thougts on that?
Also, in order to decrease some commit churn I had, I have a couple of additional commits that I could queue up:

Hound churn:

adds rubocop to Gemfile
adds a .rubocop.yml that, according to Hound (aside: I don't think that assertion is completely accurate...), is used for style-copping

Travis churn (multi-ruby):

adds tasks to Rakefile to facilitate testing locally across all rubies defined in .travis.yml

I'd be happy to create PRs for either/both or just append the commits to this, at your discretion.

Didn't see any tests to handle modifications to bin/tmuxinator; thougts on that?

We really need to get some integration tests setup as suggested in #336 to catch things like that.
As for Hound I'm finding it far too noisy really. What do you think about just having Rubocop run in travis instead? And having a rake task to just run rspec + integration tests if any + rubocop across Rubies would be great.
Be good to have separate PRs for whatever you want to tackle. I'll merge this PR now.

WRT Hound, I agree, it's pretty noisy.  I like plain ol' rubocop as you suggested.  The rubocop configuration that Hound uses by default is a contributor to that noisiness; in particular, I think the 80-character line limit in test suites is very annoying.  I'll PR what I have as a strawman to start the discussion.

Strawman PR for rubocop

Doesn't #203 do the same thing?


integer out of range converting

I have a problem during linearization with a very large pdf (2.4GB) that contains a very large number of annotations.
qpdf: integer out of range converting<PHONE_NUMBER> from an 8-byte signed type to a 4-byte signed type
this pdf is obtained as a concatenation of about 1400 small pdfs (using pdftk), which were annotated using the pymupdf (fitz) library.
I get no error when I linearize the pdf resulting from the union of the 1400 pdfs without the annotations.

pdf.json.zip

Thanks. I've gotten behind on qpdf, but I hope to have a chance to get caught up on recent issues soon.

The linearization code in qpdf is about the oldest part of the code and needs some significant rework, which I hope to do at some point. In the meantime, there are a lot of places in the code that use 32-bit integers. I'll have to review the spec to see whether that is because of bad choices I made 20 years ago or whether it is because of restrictions from the spec. It is unlikely that I will get to this soon, but I hope to get to it. There is some other effort going on to refactor the linearization code, and it might make sense to revisit that after that effort is completed.

I had a quick look at linearizing big files. The problem appears to be with the 'first object offset' in the outline hint table, which is likely to be approximately the file size (if it is in part 9) and, to a lesser extent, in the shared object hint table.
Both of these fields are 32 bit according to the PDF spec, which seems to put an upper limit to how large linearized files can get. That said, qpdf converts the offsets to int on the way, thus losing 1 bit. I think that avoiding this conversion will make all the difference for this particular file.

I don't have a really good way to test this, but I have included what I think should fix this issue in what I will release as 11.6.3 this weekend. I would be grateful if you could test it out. This issue will close automatically when I push the fix, but feel free to reopen if the problem is not fixed.

I have verified that this doesn't break linearization on large files that don't happen to have any offsets pointing past the 2GB mark.

@fmeneghetti 11.6.3 is out if you'd like to test. Hopefully this fix did the trick. If not, then I will need to figure out a way to construct a file that reproduces the problem. I can probably do it by just creating a file with annotations and then adding stuff to it until it's between 2 and 4 GB, perhaps by adding large attachments.


roundtrip for proleptic_gregorian calendar wrong for reference dates before Julian/Gregorian transition

import cftime as cf
from cftime import DatetimeProlepticGregorian as datetime
dt1 = datetime(2020, 4, 24, 16, 15, 10)
print(dt1)
units='days since -4713-01-01 12:00'
cal='proleptic_gregorian'
print(cf.num2date(cf.date2num(dt1, units, cal), units, cal))

2020-04-24 16:15:10
2020-03-19 16:15:10

For this example, it's off by 38 days.
Correct result is obtained if reference date year changed to 1583.

Actually, it works for reference date years back to -100.  Suspect it has something to do with the calculation of leap years for negative years.
It's a bug in _is_leap - fix is included in PR #188

closed by PR #188


add support to set kernel argument as NULL; fixes issue #828

Per Khronos OpenCL spec:

If the argument is declared to be a pointer of a built-in scalar or vector type, or a user defined structure type in the global or constant address space, the memory object specified as argument value must be a buffer object (or NULL).

But the following call causes error:
some_kernel.set_arg(0, NULL);

One way as suggested in #828 is to use
some_kernel.set_arg(0, sizeof(cl_mem), NULL);

Now it's just
some_kernel.set_arg(0, nullptr);

Maybe it's better if this supports C++11. Should we implement similar class to local_buffer (say null_buffer) to support setting kernel argument to NULL?

Coverage increased (+0.006%) to 84.025% when pulling e9300d9af8eec14b0781d3823401555e8bcde2bd on rosenrodt:set-null-kernel-argument into 36c89134d4013b2e5e45bc55656a18bd6141995a on boostorg:develop.

Maybe it's better if this supports pre-C++11. Should we implement similar class to local_buffer (say null_buffer) to support setting kernel argument to NULL?

I would rather have just the C++11 version. It's very unlikely that someone using C++03 will find and use boost::compute::null_buffer before just using set_arg(0, sizeof(cl_mem), NULL); (not mentioning that setting a buffer to NULL is not a popular use case).

We could stick to C++11. The trade-off would be that people using pre-C++11 won't be able to write nice things like kernel.set_args(b0, b1, nullptr).
If that's indeed where we want to go then I'll update the PR.

pre-C++11 you don't have variadic templates, you can't write kernel.set_args(b0, b1, nullptr) in C++03.

I'm sold. I'll just stick with C++11 version. Please review


feat: update to v0.25.0

Closes #277
ref: ramda/ramda#2319

add applyTo
remove intersectionWith
update jsdocs

Why this not merged yet?


Home page image

Do not display the image when scenario's have been created. Also, remove all the other fluff, and move the buttons to the bottom.

In addition, use a vertical layout for the included narratives.


Increase compatibility with Symfony 4.3 by using proper order of parameters

As suggested in https://github.com/bernardphp/bernard/issues/404

Thanks @holtkamp!

Thanks @acrobat!
Would this justify tagging a new PATCH version 0.13.1?

@acrobat would it be possible to tag a new patch release which includes this PR?

@holtkamp sorry I've must have missed that first comment! I will look into tagging a new release later today!

@sagikazarmark I see that there a quite a lot changes done on master. I'm not up to date on the state of the master branch, can I tag a new alpha release or is there more work to be done before a release?


Modify Emissions Schema & DTO validation to allow negative values

The following tables contain existing data with negative values so we need to modify excel/schemas/dtos to allow negatives. The min value should match the max value but allow negative.
DAILY_CALIBRATION			ZERO_MEASURED_VALUE
DAILY_CALIBRATION			UPSCALE_MEASURED_VALUE
DERIVED_HRLY_VALUE			UNADJUSTED_HRLY_VALUE
HRLY_GAS_FLOW_METER		GFM_READING
HRLY_GAS_FLOW_METER		AVG_SAMPLING_RATE
HRLY_OP_DATA				MATS_LOAD
MONITOR_HRLY_VALUE			UNADJUSTED_HRLY_VALUE
WEEKLY_SYSTEM_INTEGRITY		MEASURED_VALUE
NOTE: Audit spreadsheet has been updated with green highlight cells

https://github.com/US-EPA-CAMD/easey-content/pull/170 is also linked to this ticket.


User/kernel space split in TraceEvent

I encountered an error while analyzing trace data collected from my applications using TraceEvent library. I used the dotnet-trace tool from diagnostics to trace 32 bit .NET Core processes on Linux ARM64 machine.
I looked into the source code of the library and it turned out that TraceLog was counting all user space instruction pointers as kernel addresses as follows.
https://github.com/microsoft/perfview/blob/20fa6275c593c68bbd6e6a53c95fbf314304b48c/src/TraceEvent/TraceLog.cs#L2908-L2916
The method assumes that all addresses above 0x80000000 are kernel addresses (32 bit case). However, the user space size is not generally the same for all systems. For example, the default virtual address split is 2G/2G and 3G/1G on Windows and Linux respectively, and on my ARM64 machine, each 32 bit process has 4G user space.
Can anybody help with this? If this is old code base before the cross-platform .NET, should I fix the code?

https://www.kernel.org/doc/Documentation/arm64/memory.txt ?

@swift-kim, this is likely just old and from when .NET was Windows-only.  Can you tell me how this is showing up for you?

@brianrob I was trying to convert trace data into speedscope format using dotnet-trace, but I could see only a bunch of unresolved symbols (?!?) in the output file since the following method failed in the very beginning of reading an event log. The root cause is what I've described above. My trace data is piled up wtih instruction pointers which usually start with 0xEFxxxxxx and definitely ProcessForAddress() shouldn't return proc 0.
https://github.com/microsoft/perfview/blob/20fa6275c593c68bbd6e6a53c95fbf314304b48c/src/TraceEvent/TraceLog.cs#L8162-L8166

Trace-sample.zip
I attached my sample trace data. The archive contains 4 application traces in 3 formats. I temporarily modified the value 0x80000000 to larger one to convert original nettraces into ETLX and speedscope format properly.

It's hard to find the virtual address boundary between the user-kernel space programmatically. Can it be possibly passed to the TraceEvent library at runtime as an environment variable by user? (so that it can work for any platforms)

This issue has not received any attention, and I don't need a fix as my platform now supports aarch64 ;)


Changing maxdate minDate at runtime didn't affect mobile native datepicker min max

When changing the minDate or maxDate at runtime it doesn't affect the mobile native Datepicker.
https://jsfiddle.net/6mjk97cq/2/
It's also destroyed in browser when emulating mobile devices in chrome.
Latest version
Android / iOS

same here
the ux still show other dates.
but when chosing past date or disable dates will not be choosen.


Support: Passing CQL back to Geoserver

Hi, I am loving this library and am trying to push the boundries of what I can do with it, I dont seem to be able to create a CQL filter to pass back to geoserver, is this even possible?
I am trying to have the wfst load a subset of features from geoserver where the attribute "context" is 'foo' that way I can load only "foo" features from the WFS layer during the edit session.
Thanks for any advice or assistance.
Franco

Hello, thanks for feedback.
In future we will add more filters realization and some fixes for using its in initial data loading.
In this case "PropertyIsEqalTo" filter needed
<Filter>
   <PropertyIsEqualTo>
     <PropertyName>SomeProperty</PropertyName>
     <Literal>100</Literal>
   </PropertyIsEqualTo>
</Filter>

Thank you for your response!
can I do this now with the existing code or this will be possible in the future?

At this moment initial filter is not supported. We'll add this property in future.

Cool thank you :)

added support fo initial filter in #38 and PropertyIsEqualTo in 12f49d088de2b93f538ab3b7cce7757763cc7b14


feat(theme-docs): copy code

Inspired by docsify
Will be display when hovering the code block

now we have a new solution, check out https://saika.dev/#/reference/options#codeblockbuttons-badge-contentsaika-2130-


"ssh-keyscan": executable file not found in $PATH after upgrading from v0.13.2 to v0.14.1

Hi,
we have a pipeline running that is using ssh-based authentication to clone from a corporate git repository as described in https://github.com/tektoncd/pipeline/blob/master/docs/auth.md#ssh-authentication-git.
After upgrading from v0.13.2 to v0.14.1, the pipeline starts failing with the following log output:
invalid value "corp-git-secret=github.corp" for flag -ssh-git: ssh-keyscan error: exec: "ssh-keyscan": executable file not found in $PATH
Usage of /tekton/tools/entrypoint:
  -basic-docker value
    	List of secret=url pairs.
  -basic-git value
    	List of secret=url pairs.
  -docker-cfg string
    	Docker .dockercfg secret file.
  -docker-config string
    	Docker config.json secret file.
  -entrypoint string
    	Original specified entrypoint to execute
  -post_file string
    	If specified, file to write upon completion
  -results string
    	If specified, list of file names that might contain task results
  -ssh-git value
    	List of secret=url pairs.
  -termination_path string
    	If specified, file to write upon termination (default "/tekton/termination")
  -wait_file string
    	Comma-separated list of paths to wait for
  -wait_file_content
    	If specified, expect wait_file to have content

In the release notes of 0.14.1 I found the following paragraph:
The creds-init step is gone and its behavior is now handle by the entrypoint. This shouldn't have any impact except running as the same user as the step.
Do we now have to include ssh-keyscan (and others?) in the image that we are using for the task?
Thanks,
Fabian

oh that ain't good :sweat:
/cc @sbwsg
(this might mean we need a 0.14.2 :upside_down_face: )

Ah, yeah, that's fair.  Probably need to rollback the creds-init change.  I'll get a PR open.

@Fabian-K can you confirm whether you include the known_hosts section in your Secrets (mentioned in the auth doc).
If not then you can add this known_hosts field to immediately sidestep the problem.  ssh-keyscan should only run if you don't include a known_hosts entry in the Secret.

I think the fix to this might be to drop use of ssh-keyscan in the creds code.  I'm not sure it provides any kind of security to do that scan since Tekton will blindly trust any key that it returns.

Ah, yes. Currently, we don´t include known_hosts, adding it to the secret resolves the issue :)

I think the fix to this might be to drop use of ssh-keyscan in the creds code. I'm not sure it provides any kind of security to do that scan since Tekton will blindly trust any key that it returns.

Right, but this is a breaking change in behavior so… this should definitely not be in a bugfix release and will need to be handle as other deprecation.

Yep, agreed: https://github.com/tektoncd/pipeline/pull/2952

Thank you for the quick support & fix 🙏

So this should be fix in 0.14.2 (reverted the change), and will be fixed "differently" in 0.15 :wink:
Thanks for the report @Fabian-K :hugs:


⚠ Bofrost.fr has degraded performance

In 7d077cd, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 813 ms

Resolved: Bofrost.fr performance has improved in c84c2a9 after 2 hours.


Mobs deal 0 damage

Mobs deal 0 damage when player in cart or boat. I think there have to be an offset for the attack.

Could maybe deal half of a normal damage when in boat or cart...since they give you a bit of protection.

This is a bug inside minetest itseld and an issue has been posted  on git for minetest itself.

Hmmm..ok then..weird since MTG doesn't have mobs :) i will check it thought, thanks.

The bug is to do with attaching the player to entities like carts or boats and having no collisinobox around player when you do, so no damage can be dealt.

there must be some way to fix this. At least for a while.


Contact Customfield Page - Display issue while customfield type conversion from number to email

Describe the bug

When we change customfield type from number to url then if we edit customfield then it shows old filled data not showing new filled data

https://www.berrycast.com/conversations/7129c290-9009-564a-beaf-bee25b46060d

I believe this is fixed, but not moved


Optimizer support for encrypted federated training

Part of Epic: #224
Blocked by: #227
Acceptance criteria:

Add an example to #227 notebook showing how to use optimizers for backprop.

I want to work on this issue

Hello @galalen , it's yours! :)


How do you setup up authentication through xcode?

Description
I am helping our app developers setting up Swift with our internal Artifactory registry. It works perfectly fine with swift directly in CLI, but XCode complain that "Unexpected registry response: 401, expected 200".
It seems XCode does not use the credentials that Swift already set up.
Swift can be set up with:
swift package-registry set --global "https://artifactory.com/api/swift/swift"
swift package-registry login "https://artifactory.com/api/swift/swift" --username=MyUser
swift package resolve

This works fine, but XCode will not provide credentials for Artifactory.
How can XCode be set up with authentication against Artifactory?
Expected behavior
Swift packages are resolved with authentication against Artifactory Swift registries
Actual behavior
XCode throws an error because Artifactory does not accept anonymous read access.
Steps to reproduce

Upload package to an Artifactory registry
Add package as dependency in a local environment
Build project through Xcode

Swift Package Manager version/commit hash
Apple Swift version 5.8 (swiftlang-<IP_ADDRESS>.2 clang-14<IP_ADDRESS>.100)
Swift & OS version (output of swift --version && uname -a)
swift-driver version: 1.75.2 Apple Swift version 5.8 (swiftlang-<IP_ADDRESS>.2 clang-14<IP_ADDRESS>.100)
Target: arm64-apple-macosx13.0
Darwin Longs-MacBook-Pro.local 22.5.0 Darwin Kernel Version 22.5.0: Mon Apr 24 20:52:24 PDT 2023; root:xnu-8796.121.2~5/RELEASE_ARM64_T6000 arm64

Seems this is already a known issue https://github.com/apple/swift-package-manager/issues/6395


Importing files without default export

I have ts file with following imports (I just need to import and run that .ts script, it has no exports)
file.ts
import '../setup1'
import '../setup2'

It results in
file.js
import undefined from '../setup1.js';
import undefined from '../setup2.js';

which is obviously wrong and running this will fail as we are having import with the same name, it should result into this
file.js
import '../setup1.js'
import '../setup2.js'

I'm having the same issue. In my case these are files that are imported for the side effects (web-components). @Gin-Quin any chance you could have a look?

I strongly recommend to not use tsc-eam and instead to follow recent Typescript guidelines (starting from Typescript 4.7) to add the extension name (".js") at the end of the import.
This is future-proof (unlike tsc-esm).
Anyway I will look at this bug when I have some time.

@Gin-Quin I've made a PR for this: https://github.com/digital-loukoum/grubber/pull/4
The problem is that even with TS 4.7.4, webpack complains about the extensions so at the time being something like tsc-esm is the best choice.

@Gin-Quin that guideline is still wrong in my and many others opinion - you are suppose to import file which does not exist in your file system before transpilation

@joekukish I am currently solving it with a simple script which I run after tsc-esm:
import replace from 'replace-in-file';

const importsFix = {
  files: ['dist/**/*.js'],
  from: /import undefined from/g,
  to: 'import',
};

@Gin-Quin that guideline is still wrong in my and many others opinion - you are suppose to import file which does not exist in your file system before transpilation

I agree with you, importing a yet non-existant ".js" file feels wrong. It would have made more sense to add the ".ts" extension and then the compiler would transform into ".js".
But the typescript team is extremely reluctant to perform transformations. They want to stick to the Javascript language as much as possible. I think it's a mistake because they already do transformations: whenever you define an enum for example.
I know there is this proposal to make Typescript types accepted (ignored) by the browser (and by extension, Node). That would be huge because you would only write Javascript files, no more Typescript files. It would be a complete merge of the two languages.
If this happens (and I guess it's their long-term plan) then it makes sense to write the ".js" extension.
But before we get to this point, I've got used to add this weird ".js" extension. My imports are automatically done 90% of the time so I'm not even writing them myself. Also it feels safer not telling the compiler "guess which file I'm talking about", the extension removes an ambiguity. Especially in a front-end world where you can also import css, jpeg, html, etc...
I'm in holidays so I can't resolve this for now, but I'll do it when I'm back.

@Gin-Quin the problem is that the whole ecosystem is still fragmented regarding this. I've tried adding .js to every import to please browsers (so I can run tachometer comparisons against main), and then webpack complains about the extensions.

Oh, I didn't know about Webpack (I'm a Vite guy). It should work since<EMAIL_ADDRESS>👍


Remove composer.json

@bseeger @birkland  I shouldn't have committed this file, right? Sorry!

Hi @little9  the composer.json and composer.lock file should be in the /codebase directory -- so I don't think there should be either of those files at this directory level.


Change readme.dev & contributing to link to TulipaEnergyModel

What and Why
These common docs we just want to keep/update in 1 place.
Say something like "We follow the same guidelines as our main repo, TulipaEnergyModel(url)."
Possible Drawbacks
No response
Related Issues
No response

By using BestieTemplate in the repo, this is no longer an issue, see #34


Latest ENU Standard.

The indigo-devel branch is updated to follow the ENU standards rather than the old NED.
Interfacing it with robot_localization becomes much easier now.

Great! Thank you.


Sync fork with upstream

This include a lot of VirtualizedList improvements and fixes. Notably this fixes an issue with inverted FlatList that forced us to use batchSize=1.
Upstream now also includes the changes to add onStartReached.

Thanks for opening this @janicduplessis. I'm going to use this issue to list out the custom changes in our react-native-web fork to see which are outstanding, which have been merged upstream, and which we're even using.

Everything merged cleanly except for VirtualizedList, which I made sure to apply https://github.com/Expensify/react-native-web/commit/05410db4e50e2cbbee84e6b2947a9e2a8a17a353 which I think was the only change that was not upstreamed.


chore: Upgrade to pnpm 9

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 95.833%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
470

Relevant Lines:
474

💛  - Coveralls


New version: EvoDynamics v0.18.1

Registering package: EvoDynamics
Repository: https://github.com/kavir1698/EvoDynamics.jl
Created by: @kavir1698
Version: v0.18.1
Commit: b9a336d12bf617a3ad61dee35eb978ffc8d9675f
Reviewed by: @kavir1698
Reference: https://github.com/kavir1698/EvoDynamics.jl/commit/b9a336d12bf617a3ad61dee35eb978ffc8d9675f#commitcomment-130076314
Description: Simulation of eco-evolutionary dynamics on multi-trait networks

skip this


Challenge completed: Basic

https://github.com/warrenv/rejection
This is live at: http://rejection.straydog.to/

Thanks Eric!


Differences between per-hand-created and Dockerfile-created container: Potential permission issues?

I have some troubles installing a dockerized version of the mysql-cluster. At the first sight it seems that there were some problems with the cluster itself, but after some tests I think this problem is docker-related.
Sadly the error is hard to explain because there is no docker error message, its more a strange behavior of the container itself. Futhermore the given error message by the mysql-cluster is not very explanatory, too. Sorry.
Szeanrio: Create two ubuntu containers and install the mysql-cluster. Create the first one with a blank ubuntu image and install everthing by hand, for the second container use a Dockerfile. On the "blank" machine the cluster works; on the "Dockerfile" machine it doesn't.
The steps to reproduce this issue are very long, so I added them to the end of this issue!
Basically the mysql-cluster cannot create any tables and throws an error Nr. 13, which inidicates (in theory) problems with permissions. The error itself is described here.
My question: If I create two containers, one by hand and the other one with a Dockerfile using the exact same steps (except WORKDIR), are there any differences between these two at the end (in terms of usage)?
I know there are some docker-internal differences between them (each step creates a new layer when using a Dockerfile), but can a Dockerfile-created container cause any permission-related misbehavior?
Used version (on Ubuntu 16.04 LTS):
Client:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.1
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   23cf638
 Built:        Thu Aug 18 05:33:38 2016
 OS/Arch:      linux/amd64

To reproduce my issue:
Create the "blank" machine:
$ docker run -ti --name=ubuntu ubuntu

Inside the container run the following:
# apt-get update && apt-get install -y \
    build-essential \
    perl \
    libaio1 \
    libaio-dev

# apt-get install -y wget
# cd /var/tmp
# wget http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.4/mysql-cluster-gpl-7.4.12-linux-glibc2.5-x86_64.tar.gz

# groupadd mysql
# useradd -g mysql -s /bin/false mysql
# tar -C /usr/local -xzvf /var/tmp/mysql-cluster-gpl-7.4.12-linux-glibc2.5-x86_64.tar.gz
# rm -rf /var/tmp/mysql-cluster-gpl-7.4.12-linux-glibc2.5-x86_64.tar.gz
# ln -s /usr/local/mysql-cluster-gpl-7.4.12-linux-glibc2.5-x86_64 /usr/local/mysql

# cpan File::Copy Sys::Hostname Data::Dumper
# cd /usr/local/mysql
# scripts/mysql_install_db --user=mysql
# chown -R root .
# chown -R mysql data
# chgrp -R mysql .

# mkdir -p /var/lib/mysql-cluster/

# apt-get install -y vim
# vim /etc/my.cnf

Add this:
[mysqld]
# Options for mysqld process:
ndbcluster                      # run NDB storage engine

[mysql_cluster]
# Options for MySQL Cluster processes:
ndb-connectstring=<IP_ADDRESS>  # location of management server

Then create managers config:
# vim /var/lib/mysql-cluster/config.ini

Add this:
[ndbd default]
# Options affecting ndbd processes on all data nodes:
NoOfReplicas=1
DataMemory=80M
IndexMemory=18M

[tcp default]
# TCP/IP options:
portnumber=2202

[ndb_mgmd]
# Management process options:
hostname=<IP_ADDRESS>
datadir=/var/lib/mysql-cluster

[ndbd]
# Options for data node "B":
hostname=<IP_ADDRESS>
datadir=/usr/local/mysql/data

[mysqld]
# SQL node options:
hostname=<IP_ADDRESS>

Then start the manager:
# cd /usr/local/mysql/bin/
# ./ndb_mgmd --config-file=/var/lib/mysql-cluster/config.ini --nodaemon

Open 3 more terminals and connect to container
$ docker exec -ti ubuntu bash

# ./usr/local/mysql/bin/ndbd --nodaemon

# ./usr/local/mysql/bin/mysqld_safe

# cat /usr/local/mysql/data/xxxxxx.err // check if cluster runs properly

The log should say something like this (at the end):

(...)
2016-08-30 14:15:33 182 [Note] NDB Binlog: starting log at epoch 155/9
2016-08-30 14:15:33 182 [Note] NDB Binlog: Got first event
2016-08-30 14:15:33 182 [Note] NDB Binlog: ndb tables writable
2016-08-30 14:15:33 182 [Note] NDB Binlog: Startup and setup completed

Create the failing machine:
$ vim Dockerfile

Add the steps you did in the first place inside the Dockerfile:
FROM ubuntu

RUN apt-get update && apt-get install -y \
        build-essential \
        perl \
        libaio1 \
        libaio-dev

RUN apt-get install -y wget
WORKDIR /var/tmp
RUN wget http://dev.mysql.com/get/Downloads/MySQL-Cluster-7.4/mysql-cluster-gpl-7.4.12-linux-glibc2.5-x86_64.tar.gz

RUN groupadd mysql
RUN useradd -g mysql -s /bin/false mysql

RUN tar -C /usr/local -xzvf /var/tmp/mysql-cluster-gpl-7.4.12-linux-glibc2.5-x86_64.tar.gz
RUN rm -rf /var/tmp/mysql-cluster-gpl-7.4.12-linux-glibc2.5-x86_64.tar.gz

RUN ln -s /usr/local/mysql-cluster-gpl-7.4.12-linux-glibc2.5-x86_64 /usr/local/mysql

RUN cpan File::Copy Sys::Hostname Data::Dumper

WORKDIR /usr/local/mysql
RUN scripts/mysql_install_db --user=mysql
RUN chown -R root .
RUN chown -R mysql data
RUN chgrp -R mysql .

COPY ./my.cnf /etc

RUN mkdir -p /var/lib/mysql-cluster/
COPY ./config.ini /var/lib/mysql-cluster/

ENV PATH=/usr/local/mysql/bin:$PATH

EXPOSE 3306

ENTRYPOINT ["ndb_mgmd", "--config-file=/var/lib/mysql-cluster/config.ini", "--nodaemon"]

Add the files my.cnf and config.ini to the Dockerfiles dir:
my.cnf:
[mysqld]
# Options for mysqld process:
ndbcluster                      # run NDB storage engine

[mysql_cluster]
# Options for MySQL Cluster processes:
ndb-connectstring=<IP_ADDRESS>  # location of management server

config.ini:
[ndbd default]
# Options affecting ndbd processes on all data nodes:
NoOfReplicas=1
DataMemory=80M
IndexMemory=18M

[tcp default]
# TCP/IP options:
portnumber=2202

[ndb_mgmd]
# Management process options:
hostname=<IP_ADDRESS>
datadir=/var/lib/mysql-cluster

[ndbd]
# Options for data node "B":
hostname=<IP_ADDRESS>
datadir=/usr/local/mysql/data

[mysqld]
# SQL node options:
hostname=<IP_ADDRESS>

Build and try everthing:
$ docker build -t broken .
$ docker run --name broken broken

Again, open 3 other terminals to start the other processes and to check the log:
$ docker exec -ti broken bash

# ndbd --nodaemon

# mysqld_safe

# cat /usr/local/mysql/data/xxxxxx.err // check if cluster runs properly

On this machine the logs should show the following:

(...)
2016-08-30 14:23:36 599 [Note] NDB: Creating mysql.ndb_schema
2016-08-30 14:23:36 599 [ERROR] NDB: Query 'CREATE TABLE IF NOT EXISTS mysql.ndb_schema ( db VARBINARY(63) NOT NULL,name VARBINARY(63) NOT NULL,slock BINARY(32) NOT NULL,query BLOB NOT NULL,node_id INT UNSIGNED NOT NULL,epoch BIGINT UNSIGNED NOT NULL,id INT UNSIGNED NOT NULL,version INT UNSIGNED NOT NULL,type INT UNSIGNED NOT NULL,PRIMARY KEY USING HASH (db,name) ) ENGINE=NDB CHARACTER SET latin1' failed, error: 1005: Can't create table 'ndb_schema' (errno: 13)
2016-08-30 14:23:37 599 [Note] NDB: Creating mysql.ndb_schema
2016-08-30 14:23:37 599 [ERROR] NDB: Query 'CREATE TABLE IF NOT EXISTS mysql.ndb_schema ( db VARBINARY(63) NOT NULL,name VARBINARY(63) NOT NULL,slock BINARY(32) NOT NULL,query BLOB NOT NULL,node_id INT UNSIGNED NOT NULL,epoch BIGINT UNSIGNED NOT NULL,id INT UNSIGNED NOT NULL,version INT UNSIGNED NOT NULL,type INT UNSIGNED NOT NULL,PRIMARY KEY USING HASH (db,name) ) ENGINE=NDB CHARACTER SET latin1' failed, error: 1005: Can't create table 'ndb_schema' (errno: 13)

So this machine is unable to create a table inside the cluster, which indicates (according the mysql-cluster docs) that the cluster isn't setup correctly. The error inidicates a permission issue.
All permissions on the machine are exactly the same.
I really appreciate your help, because I'm clueless about this issue. Do I miss something obvious?
Thanks in advice!

This looks suspicious:
# chown -R root .
# chown -R mysql data
# chown -R mysql .

Indeed there are some weird permissions issues that crop up with AUFS where permissions change on two different layers.
Also not sure why you would chown everything to root, then later chown everything to mysql.

Thank you for your reply!  The third line should look like this:
chgrp -R mysql .
I did this in my previous tests, which also failed. I will update the issue soon.

Am 31.08.2016 um 16:02 schrieb Brian Goff<EMAIL_ADDRESS>This looks suspicious:
chown -R root .
chown -R mysql data
chown -R mysql .
Indeed there are some weird permissions issues that crop up with AUFS where permissions change on two different layers.
Also not sure why you would chown everything to root, then later chown everything to mysql.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

I did not resolve the error itself, but I got it working using less layers (i think):
So these are the working Dockerfiles:
mysql daemon:
FROM ubuntu

RUN apt-get update && \
    apt-get install -y libcam-pdf-perl build-essential libaio1 libaio-dev && \
    cpan File::Copy \
         Sys::Hostname \
         Data::Dumper

// .. install mysql cluster files to /usr/local/mysql

WORKDIR /usr/local/mysql

RUN groupadd mysql && \
    useradd -g mysql -s /bin/false mysql && \
    chown -R root . && \
    chown -R mysql data && \
    chgrp -R mysql . && \
    scripts/mysql_install_db --user=mysql

ENV PATH=/usr/local/mysql/bin:$PATH

USER mysql

ndbd:
FROM ubuntu

// .. install mysql cluster files to /usr/local/bin

RUN chmod +x /usr/local/bin/ndb* && \
    mkdir -p /usr/local/mysql/data

ndb_mgm:
FROM ubuntu

// .. install mysql cluster files to /usr/local/bin

RUN chmod +x /usr/local/bin/ndb_mgm* && \
    mkdir -p /var/lib/mysql-cluster


chore(main): release 0.9.0

:robot: I have created a release beep boop
0.9.0 (2023-02-17)
Features

analytics: model_stub for web (419c374)
analytics: Placeholder (6803231)
mine: changelog in about (b945a10)
mine: feedback (bf27d18)
update AboutListTile (8220546)

Reverts

Revert "fix: update base href" (31143df)

Tests

update constants test (77a8f22)

Miscellaneous Chores

update app icon (600755a)
update app icon again (138beed)
update app icon, again (ef06eca)

Build System

flutter pub add --dev build_web_compilers (ec67f32)
flutter pub add --dev flutter_launcher_icons (eb3d4d5)
flutter pub add --dev flutter_native_splash (f736880)
flutter pub add flutter_markdown (d6ec7c3)
flutter pub add url_launcher (b967819)
flutter pub remove flutter_native_splash (0aabf33)

Continuous Integration

build: add permissions for deploy-pages (0fb1a4f)
build: build web (817520b)
build: release artifacts (c80cef0)
build: release-web (e5f42f8)
build: set base-href (87cf500)
cache: true (e0cf1d8)

Documentation

remove todos (d9383b5)

Styles

dart fix --apply (2c467ec)
reformat by astyle (4cf2b55)
restyle by bot (cfbd3b5)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/CCXXXI/ecg_monitor/releases/tag/v0.9.0 :sunflower:


Diverging / poorly matched neural rates at simulation time

MLE or weakly penalized estimators for GLMs are known to perform poorly when generating new spike trains recurrently.
If we want yo provide tutorial on fitting and simulating spikes we should keep this in mind. Here a paper that has done some interesting work on the problem:
https://proceedings.neurips.cc/paper_files/paper/2020/file/186b690e29892f137b4c34cfa40a3a4d-Paper.pdf

Discussed the issue with Memming:
GLM not well behaved in simulaiton, especially when fitted as 1-step ahead prediction. He doesn't have a simple solution for that:

the neurips above doesn't have guarantees and computationally expansive.
my former colleague David has another with Memming on the topic paper, the method is also computationally expensive.

Generally, Memming's advice is to avoid using GLMs for forecasting

I agree with Memming's take. Maybe we could even remove the simulate
feature.
On Fri, Nov 17, 2023, 6:58 AM Edoardo Balzani @.***>
wrote:

Discussed the issue with Memming:
GLM not well behaved in simulaiton, especially when fitted as 1-step ahead
prediction. He doesn't have a simple solution for that:

the neurips above doesn't have guarantees and computationally
expansive.
my former colleague David has another with Memming on the topic
paper https://ieeexplore.ieee.org/document/8008426, the method is
also computationally expensive.

Generally, Memming's advice is to avoid using GLMs for forecasting
—
Reply to this email directly, view it on GitHub
https://github.com/flatironinstitute/nemos/issues/46#issuecomment-1815379503,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAE3NUN6QA3SSQBP4HAK7Z3YE2D67AVCNFSM6AAAAAA5IFO5LSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJVGM3TSNJQGM
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Honda Pedal Fixes

This pull request fixes a long-standing issue with Honda Pedal users. When using a pedal currently, the cruise cancel command is still sent which causes the brake to oscillate between values rapidly causing a noticeable grinding noise at low speeds. This is fixed by preventing the cancel command to be sent if a pedal is present.
I've also copied the code to limit max gas velocity that the Toyota interface file uses which greatly prevents lurching at low speeds.
Lastly, I've added some pedal tuning for the Civic as it has the worst tune out of all Hondas thanks to the higher value longitudinal tune. Without the tune it would floor it, slam brakes, and repeat until reaching around 35mph.

I removed the Civic specific tuning as I'm not sure this is the best way to go about tuning comma pedal. If accel overshoot is prioritized, the braking force is too weak to stop the car in time, so I was left with a middle where it accels pretty abruptly but brakes too much then creeps forward to meet the lead car in stop-and-go. It could need a gas delay like steer actuator delay since the car takes a second to respond most likely causing the loop to overshoot, but this will need to be a separate pull request after some more digging.
Here's a route with the changes in this PR - I do believe both of the changes in here will make the pedal experience much better for Honda owners (getting rid of the grinding is a huge win!)
Let me know if you need anything else from me to get this PR merged.

Thanks for the comment George, that was a really hacky fix now that took a closer look at it.
The root of the problem was pcm_cancel_cmd being set to true every update in carcontroller because of ret.enableCruise being set to false in interface when a gas interceptor was detected. With this being true always, the carcontroller operates like normal thus the brakes don't think they are being constantly canceled by cruise, so they no longer grind.
Cabana with pedal - no more brake grinding and pcm_cancel_cmd operating normal again
Cabana without pedal - still functions as expected, no issues


Safe-Contracts Deployed on Orderly-Sepolia

ChainId
4460
Chainlist
https://chainlist.org/chain/4460
Contract deployed
I confirm that the contracts are deployed
Version of contracts deployed
v1.3.0-libs.0
Relevant information

RPC endpoint: https://l2-orderly-l2-4460-sepolia-8tc3sd7dvy.t.conduit.xyz/
Block explorer: https://testnet-explorer.orderly.org/

Issue open: https://github.com/safe-global/safe-deployments/issues/305


There are no graphics in my blog

If I understood correctly, the "public" folder contains the required files for upload. When I upload them into my server, all I see is just plain text, no graphics, no colors nothing, like the CSS is not loading or something.
My workflow:
hugo new site myblog
cd myblog
hugo new about.md
git clone --recursive https://github.com/spf13/hugoThemes themes
hugo server --theme=hyde --buildDrafts
At the point, I have a local server, that renders the blog perfectly. All graphics, colors, fonts are absolutely fine.
the question is, how can I upload this blog on my server? I've tried by uploading the "public" folder, but no luck...

In general, you will get faster answers on http://discuss.gohugo.io/

This question was reposted and answered here:
http://discuss.gohugo.io/t/how-to-upload-my-hugo-blog-from-localhost-to-my-server/709


[Question] build custom template

Description
Is there any documentation to create our own custom template instead using default adminlte template?

@yogihardi There is no documentation to build a new custom template for detail now. You can see the default adminlte template code for reference.
https://github.com/GoAdminGroup/themes/blob/master/adminlte/adminlte.go
https://github.com/GoAdminGroup/themes/blob/master/adminlte/Makefile


Multiple chief fixes.

r+

Changes Unknown when pulling 64943b51ef48239793b41680e2fc2ec168beaaec on johngian:fix-chief-deployment into ** on mozilla:master**.


There are two black holes in the Antarctic and the North Pole.

Describe the bug
There are two black holes in the Antarctic and the North Pole.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Thanks @songsgroup for your issue.
This will be fixed by: https://github.com/heremaps/harp.gl/pull/653

This should be fixed, please check: https://www.harp.gl/docs/master/examples/#getting-started_globe-projection.html


indexes related error from xarray v0.14.0 when calling esmlab.resample(ds, freq='ann')

When I the commands
import xarray as xr
import esmlab
ds = xr.open_dataset('/glade/work/klindsay/analysis/CESM2_coup_carb_cycle_JAMES/tseries/FG_CO2_ocn_piControl_00.nc')
ds_ann = esmlab.resample(ds, freq='ann')

I get an error with the following traceback
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES/lib/python3.7/site-packages/esmlab/core.py", line 780, in resample
    weights=weights, method=method
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES/lib/python3.7/site-packages/esmlab/core.py", line 556, in compute_ann_mean
    computed_dset = dset.apply(weighted_mean_arr, wgts=wgts)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES/lib/python3.7/site-packages/xarray/core/dataset.py", line 4140, in apply
    for k, v in self.data_vars.items()
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES/lib/python3.7/site-packages/xarray/core/dataset.py", line 4140, in <dictcomp>
    for k, v in self.data_vars.items()
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES/lib/python3.7/site-packages/esmlab/core.py", line 547, in weighted_mean_arr
    (darr * wgts).resample({self.time_coord_name: 'A'}).sum(dim=self.time_coord_name)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES/lib/python3.7/site-packages/xarray/core/dataarray.py", line 2503, in func
    self, other = align(self, other, join=align_type, copy=False)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES/lib/python3.7/site-packages/xarray/core/alignment.py", line 298, in align
    "indexes along dimension {!r} are not equal".format(dim)
ValueError: indexes along dimension 'time' are not equal

Output of esmlab.show_versions()

INSTALLED VERSIONS
commit: None
python: 3.7.3 | packaged by conda-forge | (default, Jul  1 2019, 21:52:21)
[GCC 7.3.0]
python-bits: 64
OS: Linux
OS-release: 3.10.0-693.21.1.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
esmlab: 2019.4.27.post40
xarray: 0.14.0
pandas: 0.25.2
numpy: 1.17.2
scipy: 1.3.1
xesmf: 0.2.1
cftime: <IP_ADDRESS>
dask: 2.6.0
distributed: 2.6.0
setuptools: 41.4.0
pip: 19.3.1
conda: None
pytest: None
IPython: 7.8.0
sphinx: None

@andersy005 and I were working through https://github.com/NCAR/intake-esm-datastore/issues/37 this afternoon and eventually ended up at the same error. With #156 I no longer got the error in my notebook, so I think it's pretty promising for this issue as well.

Applying #156 does enable ds_ann = esmlab.resample(ds, freq='ann') to run with no error. However, the resulting dataset looks odd to me:
>>> ds_ann
<xarray.Dataset>
Dimensions:     (d2: 2, region: 4, time: 1200, year: 1200)
Coordinates:
  * region      (region) object 'Global' 'SH_mid_lat' 'low_lat' 'NH_mid_lat'
    time        (year) object 0001-07-01 17:05:00 ... 1200-07-01 17:00:00
  * year        (year) int64 1 2 3 4 5 6 7 ... 1195 1196 1197 1198 1199 1200
Dimensions without coordinates: d2
Data variables:
    FG_CO2      (time, region) float64 -0.165 0.1448 -0.9223 ... -0.9603 0.612
    time_bound  (d2, time) object 0001-01-01 01:59:59 ... 1201-01-01 00:00:00
Attributes:
    history:  \n2019-10-25 09:01:22.862617 esmlab.resample(<DATASET>, freq="a...

There is a new dimension, year, with an associated coordinate. time is somehow associated with year, instead of time. I don't know what it means that time is lacking an * in the list of coordinates for the dataset. Additionally, if I attempt to write the generated dataset to a file, I get an error from xarray:
>>> ds_ann.to_netcdf('foo.nc')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/core/dataset.py", line 1536, in to_netcdf
    invalid_netcdf=invalid_netcdf,
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/backends/api.py", line 1071, in to_netcdf
    dataset, store, writer, encoding=encoding, unlimited_dims=unlimited_dims
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/backends/api.py", line 1117, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/backends/common.py", line 293, in store
    variables, attributes = self.encode(variables, attributes)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/backends/common.py", line 382, in encode
    variables, attributes = cf_encoder(variables, attributes)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/conventions.py", line 746, in cf_encoder
    _update_bounds_encoding(variables)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/conventions.py", line 424, in _update_bounds_encoding
    "'{0}' before writing to a file.".format(v.name, attrs["bounds"]),
AttributeError: 'Variable' object has no attribute 'name'

I'm also unable to select a slice along the time dimension in the generated dataset.
>>> ds_ann.sel(time=slice('0801-01-01', '1001-01-01'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/core/dataset.py", line 2000, in sel
    self, indexers=indexers, method=method, tolerance=tolerance
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/core/coordinates.py", line 392, in remap_label_indexers
    obj, v_indexers, method=method, tolerance=tolerance
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/core/indexing.py", line 261, in remap_label_indexers
    idxr, new_idx = convert_label_indexer(index, label, dim, method, tolerance)
  File "/glade/work/klindsay/miniconda3/envs/CESM2_coup_carb_cycle_JAMES_tst/lib/python3.7/site-packages/xarray/core/indexing.py", line 140, in convert_label_indexer
    "cannot use a dict-like object for selection on "
ValueError: cannot use a dict-like object for selection on a dimension that does not have a MultiIndex

The updated #156 enables me to select a slice along the time dimension on the dataset returned by esmlab.resample.
I'm also able to write the result to a netCDF file, if I run the following prior to calling to_netcdf
for key in ['units', 'calendar']:
    ds_ann.time.encoding[key] = ds.time.encoding[key]

It looks like esmlab.resample does not propagate these encoding settings that to_netcdf relies on. That might deserve an esmlab issue of its own.
Thanks for the quick work on this @andersy005

I just fixed the encoding propagation issue as well:
In [1]: import xarray as xr

In [2]: ds = xr.open_dataset('/glade/work/klindsay/analysis/CESM2_coup_carb_cycle_JAMES/tseries/FG_CO2_ocn_piControl_00.nc')

In [3]: ds_ann = esmlab.resample(ds, freq='ann')

In [4]: import esmlab

In [5]: ds_ann = esmlab.resample(ds, freq='ann')

In [6]: ds_ann
Out[6]:
<xarray.Dataset>
Dimensions:     (d2: 2, region: 4, time: 1200)
Coordinates:
  * region      (region) object 'Global' 'SH_mid_lat' 'low_lat' 'NH_mid_lat'
  * time        (time) object 0001-07-01 17:05:00 ... 1200-07-01 17:00:00
Dimensions without coordinates: d2
Data variables:
    FG_CO2      (time, region) float64 -0.165 0.1448 -0.9223 ... -0.9603 0.612
    time_bound  (d2, time) object 0001-01-01 01:59:59.999999 ... 1201-01-01 00:00:00
Attributes:
    history:  \n2019-10-25 14:20:33.431614 esmlab.resample(<DATASET>, freq="a...

In [7]: ds_ann.time.encoding
Out[7]:
{'dtype': dtype('float64'),
 '_FillValue': 9.969209968386869e+36,
 'units': 'days since 0000-01-01',
 'calendar': 'noleap'}

In [8]: ds_ann.time_bound.encoding
Out[8]:
{'dtype': dtype('float64'),
 '_FillValue': 9.969209968386869e+36,
 'units': 'days since 0000-01-01',
 'calendar': 'noleap'}

In [9]: ds_ann.sel(time=slice('0801-01-01', '1001-01-01'))
Out[9]:
<xarray.Dataset>
Dimensions:     (d2: 2, region: 4, time: 200)
Coordinates:
  * region      (region) object 'Global' 'SH_mid_lat' 'low_lat' 'NH_mid_lat'
  * time        (time) object 0801-07-01 17:00:00 ... 1000-07-01 17:00:00
Dimensions without coordinates: d2
Data variables:
    FG_CO2      (time, region) float64 0.1078 0.3866 -0.9026 ... -0.9721 0.6262
    time_bound  (d2, time) object 0801-01-01 00:00:00 ... 1001-01-01 00:00:00
Attributes:
    history:  \n2019-10-25 14:20:33.431614 esmlab.resample(<DATASET>, freq="a...

In [10]: ds_ann.to_netcdf("/glade/u/home/abanihi/scratch/foo.nc")

Thanks!

I was running into the same issue (losing the encoding on the time dimension), and using @andersy005's latest master commit is working great for me. Thanks for pointing it out @klindsay28, I was like "oh, that's weird I need to specify a time index instead of a year" but didn't connect the dots on what must've gone wrong to make that the case. Anyway, I'm back to having years on the x-axis of my plots... I think my notebook is down to just intake-esm issues at this point.


Not all logs writing to Elasticsearch

Some logs stay only in buffer and not syncs to Elastic.
We have some messages, that randomly syncing to elastic and always present in buffer.

Me too

What I've found is that if you've got multiple instances only 1 file is synced until the day ends. When the day ends it then starts syncing the other instance files. I think we need a sub-bookmark per file and the main bookmark would rotate between them.
bookmark 1:~ sub-bookmark 1:1 buffers
This would help distribute the syncing opportunity for each file.

The durable sink has some quirks. Certainly with issues from ES popping back up. See PR #127. Might need some rewrite there.

Same issue here. Is there a workaround/fix?

Workarounds if you want to have persistent buffering; use a buffer like a queue system such as RabbitMQ or use a file buffer. Combine that with a system like logstash and you can reliable submit the events to Elasticsearch.

@mivano but if we use logstash then what is the point on having the sink anyway? Is there any fix so far? In my case it sends logs randomly every 10 or 20mins... and just one log event.

we have workaroung - stop using elasticsearch sink and write to Elasticrease throught Filebeat (using Serilog Rolling file)

I also had this issue and was able to resolve it by instantiating Serilog.Core.Logger inside of a using statement. This allowed the logger to flush before closing.

Log.CloseAndFlush() worked perfectly for me

We're still using the previous release (6.3) and see logs getting regularly lost. We're considering to upgrade to the latest release (7.1). Could anyone comment whether there're any improvements in this regards in the current version. I see the codebase was significantly reworked and now it's using hopefully more mature code from Seq's sink. I see there's a Durable sink now (not sure if it was here before) but it's designed this problem or not

We are having this problem now and as we are using Kubernetes (i.e. containers) we can't use files. They disappear when the application stops. It would be very valuable to have a reliable way to force all outstanding logs to be written before the application shuts down.

Same here. We've enabled all the possible options related to duraability but still find logs missing from time to time. It's not critical for us at the moment, so we just live with it but it worries me a lot and I haven't found any decent alternative.

Added code to call Log.CloseAndFlush() from IApplicationLifetime's ApplicationStopped callback. Not sure if it helps yet, but if it does it is an OK solution.

Please let us know if it helped
On Fri, Apr 26, 2019, 19:47 Erik Wramner<EMAIL_ADDRESS>wrote:

Added code to call Log.CloseAndFlush() from IApplicationLifetime's
ApplicationStopped callback. Not sure if it helps yet, but if it does it is
an OK solution.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/serilog/serilog-sinks-elasticsearch/issues/125#issuecomment-487044513,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAMVV4M6UXXZ4P2KA47E4WTPSL2XRANCNFSM4D55NKKA
.

Does this problem still exists? It has a decent workaround?

I wanted to gain some experience before replying. Adding CloseAndFlush was definitely an improvement. Now we always get logs for successful jobs when they are done and in many cases we get the logs for failures as well (.NET calls the handler if the application crashes). It doesn't help when the application is killed. Based on our experience so far I would say that it is a decent solution.

Logging should never break or impact your application, so it might be possible that logevents are lost in favor of having a running system. Next to that, there are multiple parts; serilog, the sink, (the durable sink) and ES itself. All elements that can fail.
CloseAndFlush is a generic Serilog function that makes sure that all batched sinks are called to flush their data. So make sure to call it when you gracefully close your application.
That might not always be possible and if you have the requirement to make sure all events are persistent, then you might need to look for a sink that does not use a buffer (even the in memory one) and persist the logevent as quickly as possible to a store (like a queue). Asynchronously you can then process the events and put them in ES. As with all options; it depends on what you need to use.
There is in serilog an AuditTo functionality which makes sure that you have reliable logging. However, this sink does not support that (yet).

Have you read Lifecycle of Loggers? Log.CloseAndFlush() only works if you're logging on the global Log object. If you create local Logger objects, they are flushed when you dispose them.

According to the life cycle page:

Call Log.ForContext(...) to receive an ILogger with additional properties attached; this doesn't need any special close/flush logic, as this will be handled by the parent logger

We always use an IOC container to inject an ILogger, I think that is common practice. Thus CloseAndFlush should work.

Old issue, cleaning up

Applied the approach that @erik-wramner mentioned (Log.CloseAndFlush() from IApplicationLifetime's ApplicationStopped callback) and it works fine: now I can see logs related to graceful shutdown!
Thank you, Eric!


Don't escape right square brackets

First, I want to say thank you for the extension, which is critical for my use of Zotero and bibtex!  But just recently I've run into a problem...
When you import a paper from the arxiv, e.g. at website
http://arxiv.org/abs/1407.5694
Zotero puts
arXiv:1407.5694 [math]
in the publication field.  Better bibtex seems to be currently exporting this as a Bibtex file with
journal = {{arXiv:1407.5694 [math]}},
or
journal = {arXiv:1407.5694 [math]},
without the double braces option.  For contrast a regular bibtex export gives
journal = {{arXiv}:1407.5694 [math]},
The problem is that Latex renders the [ and ] around math as a displayed equation.  This is a new problem.  From looking at .bbl files that I produced from a Better bibtex exported .bib file awhile ago, I am guessing that the export used to be something like
journal = {arXiv:1407.5694 \ensuremath{[}math\ensuremath{]}}.
which is fine.  Let me know if you need any more information.

Hey @mmwood, could you take a look at #460? The changes made there would render your reference as
@article{Hu2014,
  abstract = {Let X be a projective variety which is algebraic Lang hyperbolic. We show that Lang's conjecture holds (one direction only): X and all its subvarieties are of general type and the canonical divisor K\_X is ample at smooth points and Kawamata log terminal points of X, provided that K\_X is Q-Cartier, no Calabi-Yau variety is algebraic Lang hyperbolic and a weak abundance conjecture holds.},
  author = {Hu, Fei and Meng, Sheng and Zhang, De-Qi},
  archivePrefix = {arXiv},
  eprint = {1407.5694},
  eprinttype = {arxiv},
  keywords = {32Q45 Hyperbolic and Kobayashi hyperbolic manifolds; 14E30 Minimal model program (Mori theory; extremal rays),Mathematics - Algebraic Geometry,Mathematics - Complex Variables,Mathematics - Differential Geometry},
  month = jul,
  note = {\url{http://arxiv.org/abs/1407.5694}},
  primaryClass = {math},
  timestamp = {2015-02-24 12:14:36 +0100},
  title = {Ampleness of Canonical Divisors of Hyperbolic Normal Projective Varieties},
  urldate = {2014-08-11},
  year = {2014}
}

would that still be OK for your case?


Cannot create chrome profile at the mounted directory

I want to pass on a chrome profile file from my host into the container and use --user-data-dir to use that profile. It works if the directory I specify in --user-data-dir is not my mounted directory, if it is then I get this error:
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot create default profile directory
And if I set a directory that is not my mounted one, then it changes on every new container and I do not want that.
I also have the BROWSER_PROFILE_DIR environmental variable set, but it doesn't change anything.

#1173 I had the same permission issue


Multi-selection ctrl+V behaves unexpectedly depending on CodeMirror internal state

In some editor, input the string

Foo

(Foo, trailing newline) and copy this string.

open https://codemirror.net/demo/sublime.html
double-click Ctrl to select it and press ctrl+d to select the next occurrence
Ctrl+v to paste

Expected: Foo and newline are inserted at both cursor positions.
Actual: Foo is inserted at the first cursor and nothing is inserted at the second.
If you do the whole procedure within a single codemirror, the expected outcome occurs.

When the number of lines in foreign (non-local) clipboard data matches the number of selections, the editor will paste one line per selection. For local data, the editor knows the exact selections that the data was made out of, so it will treat it as a single selection, even though it contains multiple lines. This is by design.

Where is this behavior documented? And what's the motivation for having this context-sensitive functionality, since I don't know of any other editor that behaves this way?
I do agree that both functionalities have their own use cases, but it would be best to have them as separate commands that each just do the one thing. It's fine to have them not be bound by default – we can do that on our end.

I don't think this behavior is documented anywhere at the moment. If I remember correctly, at some poine someone cared a lot about getting this behavior and I figured it wouldn't hurt.
Unfortunately, there isn't much room for different commands when it comes to pasting -- the browser only allows us to access the clipboard when a native paste even occurs.
Would an option to turn this (spreading N line external content pastes over N selections) off help for you?

An option to turn the behavior off would be fine as a workaround, but I do think there is room for the two commands (since as I said, this behavior is quite useful in some cases).
In particular, pasting still happens when you add modifiers on top of ctrl/cmd+v. So if there were different commands for these two behaviors, they should be assigned to ctrl+v, ctrl+shift+v, ctrl+alt+v, etc., and should still work, right?
I'm not suggesting they should be assigned in CodeMirror's keymaps, but we could assign them as such in ours.

Attached patch adds the option (pasteLinesPerSelection). Since pasting isn't a command, in the CodeMirror sense, but a regular DOM event, defining different commands doesn't work. You could, however, register a DOM handler for paste that runs before CodeMirror's handling, and sets this new option to the desired value based on the state of a modifier key. Does that work for you?

Eh, no that's too much hassle since we also have another system of commands/shortcuts that acts as a superset of CodeMirror's. We'll just set this option to false for now and if someone wants this behavior as a separate command I can point them back at this thread :)


Integration with Elastic

I've done integration with Elastic and do some local test to ensure it's right!（some logic is translated from LangChain Python to Java）.
Elasticsearch do not  support Gson. So we must have Jackson dependency.

Hi @Martin7-1 is this PR ready for review? I've seen you did multiple changes, so I decided to wait a bit...

Hi @Martin7-1 is this PR ready for review? I've seen you did multiple changes, so I decided to wait a bit...

Sure. You can review this PR at any time. It's already done. But one thing I'm not sure about is whether we should support Multi Node of Elastic right now?

I also add some check to BulkResponse Because it will not throw an exception if only a partial error is reported during Bulk


Call to undefined function GuzzleHttp\Promise\settle()

SDK version 1.1.11
File upload is failing because of an apparent issue with Guzzle promises.

Unrelated to the issue - is this package actively being maintained?

I am not sure how to use Github, really, but this worked for me. Perhaps you can commit it?

add use GuzzleHttp\Promise;
Swap out the GuzzleHttp/Promises\settle for Promise\Utils::settle

We're seeing this issue too. We had to upgrade Guzzle because other dependencies require the latest version, and we are now getting this error as well.

@tomgibb there is indeed a solution here. The problem is that it does not exist in this package.
@jake-lundberg this problem is in need of a fix. I've contact Filestack support and they have brushed it off.

That was November 2023.

I had to launch something sooner vs. later which led me to searching
github/packagist to find a fork with the proper changes. I found one that
was appropriate: https://github.com/pubpackage/filestack-php
On Wed, Jan 10, 2024 at 2:17 PM Steven Maguire @.***>
wrote:

@tomgibb https://github.com/tomgibb there is indeed a solution here.
The problem is that it does not exist in this package.
@jake-lundberg https://github.com/jake-lundberg this problem is in need
of a fix. I've contact Filestack support and they have brushed it off.
@.*** (view on web)
https://github.com/filestack/filestack-php/assets/1851973/a7133e40-41f6-4c2b-bca1-6e171b5ac1d0
That was November 2023.
—
Reply to this email directly, view it on GitHub
https://github.com/filestack/filestack-php/issues/62#issuecomment-1885522625,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AD45U3KR5TB2OPDUYIWWSTLYN3SLHAVCNFSM6AAAAAA6WOTEV2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBVGUZDENRSGU
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
347.237.2778

Thanks @tomgibb - that is a useful find. Perhaps @pubpackage is open to creating a pull request against this package? Although considering there is PR #55 from 2021 still open, getting a PR reviewed a merged may also be asking too much.


Investigate Selenium 3 compatibility

We should check if the library needs any change to be compatible with Selenium 3.
At least we'll have to add 'geckodriver' executable to the PATH for the CI tests:
https://travis-ci.org/gunesacar/tor-browser-selenium/jobs/149722181
Message: 'geckodriver' executable needs to be in PATH.

Here's a WIP branch: https://github.com/gunesacar/tor-browser-selenium/tree/tbgeckodriver

Almost everything works fine for TBB versions 6+ with Selenium 3.0.0b2.
There's a minor issue is driver.set_context, which is noted here:
https://github.com/gunesacar/tor-browser-selenium/commit/b105ce28dc18fb5f93280441d2455d5054c5b0a8#diff-7a7914417f2bac1e28f9c83336bce811R39
5.5.5, will not work with Selenium 3. We may need to drop support.
CI tests pass for version 6 TBBs, python 3.4 and 3.5 errors are due to Selenium's own bug, which should be fixed in 3.0.0b3.

Import errors in Python 3.4 and 3.5 fixed with the release of Selenium 3.0.0.b3.
https://travis-ci.org/gunesacar/tor-browser-selenium/builds/160031352

:+1: for landing Selenium 3 support.

@fowlslegs, in case it helps with for your internal scheduling, we need to wait until the release of the ESR 52 based Tor Browsers to land Selenium 3 support.
Tor Browser 6.X versions are based on ESR 45, which is not supported by geckodriver.
The expected date for ESR52 based Tor Browser is June 13: https://lists.torproject.org/pipermail/tbb-dev/2017-February/000472.html.

I tested the tbgeckodriver branch with tor-browser-crawler and the latest TBB (7.0.2). I am able to run the crawler successfully in our setting (haven't tested edge cases). However, it doesn't work for newer versions of geckodriver and Selenium.

Thanks @mjuarezm !
I guess the next step would be to check which tests fail with that branch and address them.

Here's the PR for the selenium 3 compatibility: https://github.com/webfp/tor-browser-selenium/pull/88

Fixed in https://github.com/webfp/tor-browser-selenium/pull/88#issuecomment-334484938

Getting this error message on torbrowser-linux64-7.0.8. Tested this with selenium 3.6, tbselenium 0.3.2 on Ubuntu 16.04 (Xenial64).
self.driver = TorBrowserDriver(os.environ['TBB_PATH'])

File "/home/ubunutu16/.local/lib/python2.7/site-packages/tbselenium/tbdriver.py", line 57, in init
log_path=tbb_logfile_path)
File "/home/ubunutu16/.local/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 144, in init
self.service.start()
File "/home/ubunutu16/.local/lib/python2.7/site-packages/selenium/webdriver/common/service.py", line 81, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH.

@tadatitam you need to have geckodriver executable (version 0.17.0) in your system's path starting from 0.3.2. It's required by Firefox actually.
You can download it from the following address:
https://github.com/mozilla/geckodriver/releases/download/v0.17.0/geckodriver-v0.17.0-linux64.tar.gz"
You can also check the relevant MDN documents:
https://developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver#Setting_up_the_geckodriver_executable
I will open an issue to document that requirement. Sorry about that.

@gunesacar Ah, I see. I was getting away on Firefox by specifying executable_path='path/to/geckodriver'.
Thanks to your suggestion, I was able to get Tor to launch by adding the location of geckodriver to PATH: i.e.  However, Tor fails to open any page, and throws this new error:
    self.driver = TorBrowserDriver(os.environ['TBB_PATH'])
  File "/home/ubunutu16/.local/lib/python2.7/site-packages/tbselenium/tbdriver.py", line 57, in __init__
    log_path=tbb_logfile_path)
  File "/home/ubunutu16/.local/lib/python2.7/site-packages/selenium/webdriver/firefox/webdriver.py", line 154, in __init__
    keep_alive=True)
  File "/home/ubunutu16/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 151, in __init__
    self.start_session(desired_capabilities, browser_profile)
  File "/home/ubunutu16/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 240, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/home/ubunutu16/.local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 308, in execute
    self.error_handler.check_response(response)
  File "/home/ubunutu16/.local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: connection refused

This is a generic error when the geckodriver doesn't respond for one reason or other. To debug this interface you may listen to loopback interface with Wireshark. But, this is not guaranteed to give you anything.
I'd check the geckodriver version, you need to have version 0.17.0.
If you are using custom parameters for the initialization, feel free to run your code by me.
I just checked to make sure, the below code seems to work with tbselenium 0.3.2:
https://gist.github.com/gunesacar/1b48b71d8619885a0dcb95bb59da592c

Thank you thank you. There was a mismatch in the geckodriver version. I was using 0.19.0. :)


[PriorTeamNote]-ToDo: Setup dedicated hosting

Setup dedicated hosting so that personal accounts don't have to be used.

Waiting for the database authorization

https://eecs.oregonstate.edu/education/scheduleit


[Event Request] table 5876 "Phys. Invt. Order Line" - OnBeforeCheckLine

procedure CheckLine()
var
    IsHandled: Boolean; // NEW LINE
begin
    // New code - start
    OnBeforeCheckLine(Rec, IsHandled);
    if IsHandled then
        exit;
    // New code - end
    TestField("Item No.");
    TestField("Qty. Exp. Calculated", true);
    TestQtyExpected();
    TestField("On Recording Lines", true);
    TestQtyRecorded();
end;

// New code - start
[IntegrationEvent(false, false)]
local procedure OnBeforeCheckLine(var Rec: Record "Phys. Invt. Order Line"; var IsHandled: Boolean)
begin
end;
// New code - end

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 10881.