Pagination for large dashboards

I'm probably pushing this a bit far - but why not.
I'm trying to monitor ~3500 services, which group together into ~450 groups. At this scale the UI and API are unusable, and requests take ~2.5 minutes to complete.
It'd be great to be able to do something simple to try and alleviate the burden here - I think the actual tests are working fine given the concurrency of go, and I have disabled the locking mechanism, but the UI is holding it back. This was tested with the sqlite backend (I'm currently testing the memory backend, but it takes a while to start testing the first of each service with so many to start).
Maybe these are some options to consider:

Pagination
A setting to display only services which are down (or down within X period of time)
Less comprehensive landing page (i.e. less data to render) which only displays the state currently with no history until you click that service
Group based landing page which shows only the state of the group as an aggregated entity, instead of digging into individual services

Feel free to tell me that this is way beyond the scale gatus was ever designed for, or ever planned to support.

3500?! 🤣
That's far beyond what I thought anybody would ever use Gatus for, and I can see why it the UI would be unusable. Even just the API alone must return an insanely large response. In fact, the pagination built around the API just takes in consideration the number of results to return per service, not the actual number of services to return.
Even if we just looked at the results per groups, 450 is quite a lot. I don't know if your tests are very similar to each other, but at that scale, perhaps splitting the workload between several Gatus instance would make more sense.
That being said, I do think that it would be nice to have ability to only expose the groups rather than the groups and their underlying services would be nice. In fact, #149 mentions something similar-ish.
What I have in mind would be something like this:

where it would be possible to click on the service group and list all the individual services underneath it, but through a separate API call rather than using a single call to retrieve everything.
There's obviously a couple of things to think about as well:

What about users who have <10 service groups, where it makes sense to retrieve everything in one go?
Would it be a good idea to have a page dedicated to a single service group, similarly to how there are pages for each individual service (example)
Does it make sense to assume that a lot of users will have this many services? While it's not uncommon to have hundreds of tests, when it gets to the thousands, looking at a dashboard with so many entries feels a little bit too overwhelming, in which case it would make more sense to rely on alerting to get notifications rather than look at the dashboard.

After leaving it overnight to catch up, when using the memory storage type, the UI responds within a few seconds (rather than minutes). For the application I have, memory storage is probably fine, but the UI is still too cluttered.
All of the tests are identical, each is a single HTTP endpoint, the use-case is a scaled application distributed across multiple servers with multiple health check endpoints on each server reverse-proxied to backend services. A simple 200 response is sufficient.
Definitely think exposing groups only is an excellent way forward here, and the UI layout you've drafted looks like a great way to represent that, providing as you said, it's a separate API call.
In response to your other points:

This could be a global config setting, if the top tier view is either group based or service based.
Yes, I think so, it might even be desirable to not bother collecting the detailed metrics for each service, and collect information only for the group as a whole (with a setting to correspond with that). This would probably be pretty close to #149 by the end (as the group would represent an eneity, and a service would represent a check within that entity, it's just a different hierachy structure to accomplish the same thing).
Definitely looking at the dashboard is overwhelming, but the fact gatus is able to even poll this many services reliably (with surprisingly little load on the server running it) goes to show that it's a really capable tool, and only UI changes are needed to allow it to scale to really well.

@dchidell What do you mean by this?:

I've given this some more thought, and typically the only interesting services are those experiencing problems.

But anyways, I was thinking of simply adding support for paging.
That, coupled with maybe some kind of optional sorting capability at the API level to enable things like surfacing failing services first, should be sufficient.
I don't know when I'll work on that though. I was hoping maybe I could manage it before v3.0.0 -- but we'll see.

Quick update on the above ^
I released v3.0.0 today, and while paging hasn't been implemented yet, I did make the breaking change necessary to allow paging.
Before, /api/v1/statuses returned a map of service statuses:
{
  "group1_service1": {},
  "group1_service2": {},
  "group2_service1": {}
}

but now, /api/v1/services/statuses, the replacement of the aforementioned endpoint, returns an array of service statuses:
[
  {
    "key": "group1_service1"
  },
  {
    "key": "group1_service2"
  },
  {
    "key": "group2_service1"
  }
]

This was the breaking change that was needed for paging.
I'm thinking of employing a slightly more unusual approach to paging, but before I get into my idea, let me talk about the problem.
The problem
Let's say we implement paging; we want 20 services per page.
In total, you have 38 services split into 5 groups:

group1 has 4 services
group2 has 7 services
group3 has 10 services
group4 has 15 services
group5 has 2 services

So now what do you do for your first page? The first 20 services would have all services from group1and group2, but you'd only have 9 out of 10 of the services from group3.
The solution
I think that the most transparent solution would be to implement paging that revolves around the number of groups that can be shown in one page.
I'm thinking that 10 groups per page would be a good default value, and you may be thinking "that's way too much", and that's on purpose.
Most people will have less than 10 groups on their dashboard, which means that it won't impact them, but for people like you who have thousands of services, this means that you can willingly use groups for the sake of pagination.
Caveats
This will most likely make #35 harder to implement. I initially wanted to implement #35 by using just the front end for filtering, but if we add pagination for the main dashboard, it means that the filtering would need to be done on the back-end.

@dchidell I've finally decided to look into improving the performance yesterday (couldn't reply until now due to work). I'm not sure if you still have ~3500 endpoints, but if you do, I have a couple questions for you.
Does all of your endpoints have the same interval, or do they differ? If they do differ, could you elaborate? For instance, 50% of your endpoints with an interval of 10m and 50% of your endpoints with an interval of 1h.
The best solution would likely be to fine-tune the SQL queries, but I'm considering adding a smart layer of cache on top of the persistence layer. It wouldn't really help for infrequently visited dashboards, but for frequently visited dashboard, it's a significant improvement.

We have more now, we're constantly growing! But the order of magnitude is still the same.
The endpoints are all the same interval, and all would be HTTP. Essentially we have ~1000 machines each running ~5 different docker based web services. The 5 web services are all very similar to each other, and are the same across the machines. The interval would ideally be as low as possible, to detect failures whenever they happen. Ideally 1 minute. Providing gatus can make use of multiple cores / threads it can quite happily be run on a dedicated machine with over 100 cores.

@dchidell I've just pushed the image twinproduction/gatus:experimental built from #314. It's not in the latest branch yet because I'd like if somebody properly tested it first.
The first time you load the dashboard will be a bit slow, but the subsequent ones should be significantly faster.


[Feature request] Consume-switch

Having the ability to enable/disable "consume" would be very beneficial (for me). Do you think you could add that as a switch or something? Great integration btw, I make great use of it every day :)

Hi @Switch123456789 ,
Not sure what you mean by a "Consume" switch.

Found it, it is in the API. I feared it was in iris.
It should be feasible to implement it easily...

fixed with #34

Awesome, thank you man! =)


Improve offline/partition detection

The current implementation simply checks if we've seen a block within some constant time period.
ISTM there are two separate things we're interested in when we talk about whether the node is online.
Non-adversarial network failures
For users in the testnets, they look at coda client status to figure out if they're connected properly. If their status is offline, their client is probably misconfigured, their network is down, etc. We could answer that question by querying the network layer. E.g. ask how many peers we've successfully connected to in the last 30 seconds.
Potentially adversarial network partitions
But the second thing is whether or not >50% of stake is online on the same network as the user. If the attacker controls the network then they may censor blocks arbitrarily, and cause the target to believe a state which will never be finalized. It's very plausible an attacker could do this, e.g. with a stingray, malicious WiFi, or in more extreme situations if they own or have compromised the target's ISP. It's unacceptable if I can go into a coffee shop, switch to their WiFi, sell something to a guy and see my account balance increase, then leave, switch back to cellular and see the money disappear. A less bad but still problematic scenario is where the attacker makes me think I haven't received a transaction that I have on the chain that will be final.
We can estimate the fraction of stake online in the last n slots through Bayesian inference, although the choice of hyperparameters is non-obvious since the fraction of stake that is active may rise and fall over time. An attacker may manipulate our estimate of the active stake fraction as well.
Having thought of all that it seems to me what we really want is not the probability that >50% of stake is online, but the probabilities that my best tip a) will be finalized and b) is the global best tip. We want the finalization probability for old blocks as well. This probability is equal to 1 - the probability there exists a stronger, distinct, chain. Under the relevant modelling assumptions. My intuition says there's no efficient closed form equation for that probability (at least in the presence of min_window) and we'd have to go to Monte Carlo, which is not great but might be OK.

This is closely related to both #3272 and #3225, in the former we need to know the probability a block exists that is more than max_length higher than our best tip, and the in the latter we need to know the probability that the best block we've fetched on startup will be finalized/is the global best tip.


Not work in Sven Coop and CS 1.6

Hi latest updates have viruses? I don't know how to fix it because in my antivirus haven't restore files and backup need turn off antivirus but when I again start windows then again failes removes can you fix it please because I still sit on the windows 7 and I need antivirus

it's false positive. build from source code by yourself and see if it still reports virus.

Hi I tried to compile by myself there have errors with compilling on vs2022 there writting error how I can adding to you on the steam and discord can you help me please, I want play with this mod this mod is amazing on Sven Coop and CS 1.6 and Half-Life

My discord Mysticpawn#2073 please add me


Update to latest transformer protocol & use init (Resolves #260).

Updates to the latest version of the transformer protocol.
This also changes the implementation to actually use the @@transducer/init method, which will allow users to use transformers that wrap the result object for their own purposes.

LGTM.


HTTP API Logger logs request path and method too

Fixes #1546

Codecov Report

Merging #1644 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1644      +/-   ##
==========================================
+ Coverage   98.46%   98.46%   +<.01%
==========================================
  Files          59       59
  Lines        2862     2864       +2
==========================================
+ Hits         2818     2820       +2
  Misses         44       44

Hi @ketanbhatt - Thanks! Please see my comment on your other pull request regarding our Contributor Licensing Agreement (CLA):   https://github.com/bigchaindb/bigchaindb/pull/1643#issuecomment-314151859
Dev Team:  Please don't merge this PR yet; we have to clear the CLA process first.

@TimDaub I assigned you to review this PR because you created the original issue. @ketanbhatt has now agreed to the CLA, so that's not a blocker any more.

@sbellem anything pending on my side?

@ketanbhatt - This week, @sbellem is away on vacation. Sorry for the delay.

@ttmc @TimDaub actually I had a doubt.
Earlier the format for the errors was: HTTP API error: %(status)s  - %(message)s which has now been changed to HTTP API error: %(status)s - %(method)s:%(path)s - %(message)s.
For some people, who are parsing these logs for indexing/monitoring, won't this be a breaking change? Do you think this is a valid concern? If yes, should we mention in docs somewhere?
Thanks

@ketanbhatt very good point. How about we append %(method)s:%(path)s to the log message then? /cc @r-marques

I am not sure if that would be completely foolproof as well. From what I know, appending it would still mean that a lot of custom patterns will stop recognising the logs, given how they work (example: http://grokdebug.herokuapp.com/patterns#)
I have the following ideas:

We start logging twice. Once in the old format, once in the new format, with a warning that the old format would be deprecated. And in the next version (or according to BigchainDB's deprecation policy) we remove the old format.
Let the breaking change happen, but warn in the release docs that this would break. Or maybe we wait for this change to merge until the next big release ( 😢 )

What do you think @TimDaub

For some people, who are parsing these logs for indexing/monitoring, won't this be a breaking change? Do you think this is a valid concern? If yes, should we mention in docs somewhere?

Interesting questions! I'm thinking of workarounds on how to merge this and not break things.
First, let's compare the two outputs.
Behavior in v1.0
[2017-07-25 16:06:02] [ERROR] (bigchaindb.web.views.base) HTTP API error: 404 - Not found (webapi     - pid: 14634)

Behavior in this branch
[2017-07-25 16:07:39] [ERROR] (bigchaindb.web.views.base) HTTP API error: 404 - GET:/api/v1/transactions/d3rp - Not found (webapi     - pid: 15900)

Suggested behavior
What if we slightly modify the error and unify the two things like:
[2017-07-25 16:06:02] [ERROR] (bigchaindb.web.views.base) HTTP API error: 404 - Not found GET:/api/v1/transactions/d3rp (webapi     - pid: 14634)

The string would be:
'HTTP API error: %(status)s - %(message)s %(method)s:%(path)s'

This should't break log parsers because message is a "generic string with a - in front".

Valid points, @ketanbhatt.
As an example, a current log message looks like:
[2017-07-13 14:45:22] [ERROR] (bigchaindb.web.views.base) HTTP API error: 404 - Not found (webapi     - pid: 62)
I wouldn't mind changing it to something like
[2017-07-13 14:45:22] [ERROR] (bigchaindb.web.views.base) "HTTP API, 404, GET, /xyz, Not found, webapi-pid: 62 as per:

Format the log with space as delimiter:

%(status)s %(method)s %(path)s "%(message)s"

or bunch the message in quotes and delimit using commas:
"%(status)s, %(method)s, %(path)s, %(message)s"

Log analysis usually becomes easy with space or comma delimited tokens supported implicitly in many tools (and trivial to code if needed to be done from scratch).
Also, the second option groups message after the module together in double quotes.
I am not a big fan of custom hyphens/separators with spaces around them which will need custom parsing logic. I like nicely tokenize-able strings which won't be an overhead to process.
@TimDaub I wouldn't worry too much about breaking any existing log analytics as I do not think anyone else currently parses the log (IPDB parses logs, but that won't break because of the above changes).

@vrde good suggestion. Although I would like if there was a separation between the message and the new info we will add (as a future developer who starts consuming these logs with no knowledge about how this format came to be, I will just think that this isn't a good way to output logs).
@krish7919 I was thinking so too, that the change will not break a lot of things, and nothing too critical. But quoting just the message or using , will break the patterns and be ambiguous respectively.
I think to keep all your above points in mind, and to not go too far away from the current format (by changing the format to use double quotes etc.), we can simply append - %(method)s - %(path)s to the old format (Like how @TimDaub suggested). This will have the following advantages:

Format is consistent with the old format
Will not break the majority of the parsers as they will catch message - method - path as just message, worst case.
The separation method (using -) is consistent

My only qualm with this is that the message should ideally be the last thing in the log. Thats how all other standard software logs output as well.
Keeping this in mind, I suggest the following final format:
HTTP API error: %(status)s - %(method)s:%(path)s:%(message)s
This has all the benefits of the previous method (of just appending) and also keeps the log format more consistent.
Sorry for the long comment, just thought I should also document the thought process for future reference. Let me know what you guys think and I will quickly make the desired changes.
Thanks

Sorry for the long comment, just thought I should also document the thought process for future reference. Let me know what you guys think and I will quickly make the desired changes.

This PR generated a really interesting question. Let's wait a bit to see how the team reacts to my changes for the CHANGELOG.md.

The problem with using custom delimiters (like - or : above) is that they cause a lot of problems when they appear in the message field.
The quotes and csv suggestions are just what I have seen to be widely used. It also makes it easy to write parser and/or regex for log messages.
I will approve of with any of the solution described above as long as we know the pros and cons and are not flummoxed by unexpected behavior during log parsing (might be of interest to @ttmc when configuring Azure log analytics for IPDB).

@vrde and @krish7919 --- I haven't been following this PR very closely. Is merging it still blocked by the decision about how to handle the changes to the logging message format?

@vrde and @krish7919 --- I haven't been following this PR very closely. Is merging it still blocked by the decision about how to handle the changes to the logging message format?

It isn't. We need to remember that parsing will fail in case we have any more - in the path or message fields in the format specified in the code:
logger.error('HTTP API error: %(status)s - %(method)s:%(path)s - %(message)s', request_info).
Other than being cognizant of that fact for now, I don't think anything's blocking me from this change.

@vrde are you okay with merging this pull request? If so, please feel free to merge it.


Add named keys to Press Key.

Adds named keys escaped with double backslash, for example \\ARROW_LEFT.
Also normalized test file.
Fixes #275 .

The original escaping of ASCII codes is, in my opinion, somewhat ugly and double escaping to use WebDriver keys is even uglier. Backslashes don't tell much there in general and Robot requiring them to be escaped makes the syntax really awkward.
What about using prefixes like with locators? Something like text=foo, ascii=10, and webdriver=ARROW_UP with text= being the default if there is no prefix could work. We probably could come up with better prefix names but I believe you got the point. This would also make it easy to add more ways to specify keys in the future. The old escaping mechanism to use ASCII codes should still be supported but could perhaps be deprecated.

I very much like this!

This looks good to me now. @pekkaklarck, comments?
@emanlove, see any issues?


Feature/tri 404 setup irs isolated integration test environment infrastructure

Asynchronous Job Orchestration

transfer to draft pull request


Setting color_output=no does not disable the output of color format codes

I think --color_output=no should not just use the default color for everything, it should stop color format codes from being outputted completely. The current behaviour is very misleading to me.
https://github.com/boostorg/test/blob/35c9acf5bd38d625ebd45cf940c5b9a19aa2295b/include/boost/test/utils/setcolor.hpp#L304
To understand why it is a problem let me describe my test setup:
I use ctest to run boost tests and instructed ctest to output everything to a file.
The file contains all color codes that boost writes, --color_output=yes/no only changes if its all the same color or not, the codes remain any way.

Agreed.  This also happens when trying to run unit tests within Xcode's debugger windows (which doesn't support ANSI colors).
I was able to work around it (in my particular case) by tweaking the #define BOOST_TEST_SCOPE_SETCOLOR.  I added these lines before including my main unit test headers:
#include <boost/test/utils/setcolor.hpp>
#undef BOOST_TEST_SCOPE_SETCOLOR
#define BOOST_TEST_SCOPE_SETCOLOR(is_color_output, os, attr, color)

That basically just disables ansi color altogether.

IMO, I think that if you say --enable_color=no, it should completely skip the ansi codes altogether.

A proposed patch can be found at https://github.com/boostorg/test/pull/154

Closed through PR #154


update failing test projects

The latest dependabot updates caused some build failures.
Fix them.

ping @adegeo for review


[Feature]: Add jest.requireActual to jest-mock

🚀 Feature Proposal
The jest-mock package package provides mocks for functions like jest.fn but not jest.requireActual
Motivation
I'm a guest in this environment so I don't know if this usage is correct. It's using jest-mock to silence some Jest mocks that are transitively imported into the bundle.
It has been working fine so far but a jest.requireActual has fallen into the path the bundle causes the error undefined is not a function (evaluating jest.requireActual)
Example
In a webpack config:
    new webpack.ProvidePlugin({
      document: "min-document",
      jest: "jest-mock",
    }),

Pitch
jest.fn exists, so perhaps jest.requireActual also should? 🥹

jest-mock only deals with spying and stubbing, it is not involved in module resolution directly. So it doesn't really make sense there.

What's your use case? Are you bundling jest tests to run outside of Jestm

Here's the (complex) setup as I understand it:

There's a main project and a secondary "leech" script that imports some files from the main project in order to generate a manifest
Webpack bundles this script because webpack can reuse the existing path-mapping/transpilation setup from the main project
There are some files in ./__mocks__ that are picked up automatically by Jest
The script's webpack config also uses these "mocks" to silence/ignore some APIs that are not used, as to not have to duplicate the effort
Most of these mocks are plain, like export function getApi() {return axios} but others include jest.fn

here's when jest-mocks becomes useful

Webpack uses jest-mocks to provide the jest global and silence all of these calls

😅
So I think that the desired outcome here would be for jest-mock to expose a non-functional requireActual function just like it exports jest.mock and jest.fn

It doesn't expose jest.mock either, as that's another one that interacts with module resolution.
I think it'd be cleaner to just use a module replacement in webpack itself, or maybe a babel plugin (or whatever transpiler is used - if none, you can do it with webpack plugins as well) to strip the jest. calls entirely.
jest-mock adding exports that don't work I think would just be confusing for everybody who don't have this edge use case.

just use a module replacement in webpack itself

That's what got us into this situation 😅 because of module replacement towards mocks.
I agree that either stripping the calls or providing custom/extended jest-mock would be a more reasonable solution.
Ideally the main project should be restructured to avoid this mess in the first place, but alas.

Ideally the main project should be restructured to avoid this mess in the first place, but alas.

Haha, I feel like that's often the closing comments when it comes to things like this 😅


Release 3.1.0

https://github.com/AliSoftware/Dip/wiki/Roadmap#version-310

Great, will look at that this week-end!

Ok, so,
I'm ok with the content of that release/3.1.0 branch and ready to merge.
But I tried finishing the release and got conflicts when gitflow is trying to merge the release into develop, and I have to admit I'm not totally confident with resolving those conflicts. Especially when merging release/3.1.0 into develop some lines came from 3.1.0 and others from changes made during the feature/circular-dependency feature branch.
Besides I realized that when you made your hotfix/migration-note branch I merged it via GitHub so it only merged it into master and didn't treat it as a proper git-flow hotfix, where it should also have been merged into develop too; so we had that commit missing in develop too now, and the git graph starts to be a bit messy..
Here's what I did so far

I merged the hotfix/migration-note commit into develop, like it should have been done all along if I didn't merge the PR via GitHub but via a proper git-flow process, sorry for that
I rebased feature/auto-injection and feature/named-runtimer-arguments on top of that new develop

Here's what remains to do

git-flow finish release the release/3.1.0 branch, meaning that we need to merge it into master (no problem here) and into develop (leading to conflicts i'm not confident enough to resolve, fearing it might break your work on develop)
Maybe rebase the feature/* branches on top of develop once that merge is done to have a cleaner and more understandable git graph from now on?
Check everything, as this Git graph is looking a bit like spaghettis lately so a mistake can quickly slip through. Especially check that the code in master after the 3.1.0  tag is indeed what we expect it to be and we didn't mess up the merge in the process, and that develop didn't get any regression during the merge/rebease either

No problem, I will do the merge. Will close this then and make a release.

I realize that you may not have the access right to merge to master but at least you should be able to merge to develop then I can merge to master myself.

Ok. So I'm pushed develop with this brunch merged in :tada:
Feel free to close this and make a release.

with this brunch

:egg: :smile:
Talking about brunch, will close and release that after lunch :wink:

Version 3.1.0 released and pushed to CocoaPods ! :tada: Thanks for your work on this!
You can now rebase the feature/auto-injecton branch on top of develop, and remove the feature/named-runtime-arguments branch.


Temporary Queues and Topics

I would like the facility to work with JMS temporary queues. This is so that I can then implement something like the EAI Request Response pattern atop jms4s for our own application.
Would this be something that would be welcome as a Pull Request to this project?
I thought this might be a small addition to jms4s to add this functionality... Unfortunately, I needed to add some additional API so that the Temporary Queue/Topic can be created just once and then reused for the Consumer and Producer(s).

Hi @adamretter, thanks for your contributions! I will take a look at the changes and run the tests trying to understand what's going on.
From a first look a the ibmMQ error it seems that some permissions are not set in the "scripts/definitions.mqsc"

@faustin0 All tests are passing and I hope this is now in good shape for your review... please let me know what you think?

@adamretter i'm having troubles creating a consumer (with a concurrency level >1 ) for a temp Q  that use the JmsDestination, it fails with an MQ error.
I'm not very familiar with the use of temp Q, are multiple consumers allowed for a temp Q ?
if yes, could you add a test where a consumer(for a temp Q) is created with a concurrency > 1 ? Like in the JmsClientSpec suite (a more high level test than using context directly)

hi @adamretter, as you can see, the build is failing for the new test (with the same error i was getting in my local testing):
com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2042' ('MQRC_OBJECT_IN_USE').
Isn'it failing on your local testing?

@faustin0 Sorry, I only got as far as adding the additional tests, I have not yet looking into the issue that causes the test to fail. I also still see locally this error:
[info] - publish 50 messages to a temporary queue and then consume them concurrently with local transactions *** FAILED ***
[info]   com.ibm.msg.client.jms.DetailedIllegalStateRuntimeException: JMSWMQ2008: Failed to open MQ queue 'AMQ.63BFFFB125461803'.
[info]   at com.ibm.msg.client.jms.DetailedIllegalStateException.getUnchecked(DetailedIllegalStateException.java:274)
[info]   at com.ibm.msg.client.jms.internal.JmsErrorUtils.convertJMSException(JmsErrorUtils.java:173)
[info]   at com.ibm.msg.client.jms.internal.JmsContextImpl.createConsumer(JmsContextImpl.java:420)
[info]   at jms4s.jms.JmsContext.$anonfun$createJmsConsumer$3(JmsContext.scala:95)
[info]   at blocking @ jms4s.ibmmq.ibmMQ$.$anonfun$makeJmsClient$2(ibmMQ.scala:56)
[info]   at delay @ org.typelevel.log4cats.slf4j.internal.Slf4jLoggerInternal$Slf4jLogger.$anonfun$info$4(Slf4jLoggerInternal.scala:91)
[info]   at delay @ org.typelevel.log4cats.slf4j.internal.Slf4jLoggerInternal$Slf4jLogger.isInfoEnabled(Slf4jLoggerInternal.scala:66)
[info]   at ifM$extension @ org.typelevel.log4cats.slf4j.internal.Slf4jLoggerInternal$Slf4jLogger.info(Slf4jLoggerInternal.scala:91)
[info]   at *> @ jms4s.ibmmq.ibmMQ$.$anonfun$makeJmsClient$7(ibmMQ.scala:73)
[info]   at make @ jms4s.ibmmq.ibmMQ$.makeJmsClient(ibmMQ.scala:71)
[info]   ...
[info]   Cause: com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2042' ('MQRC_OBJECT_IN_USE').
[info]   at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:203)
[info]   at com.ibm.msg.client.wmq.internal.WMQMessageConsumer.checkJmqiCallSuccess(WMQMessageConsumer.java:222)
[info]   at com.ibm.msg.client.wmq.internal.WMQMessageConsumer.checkJmqiCallSuccess(WMQMessageConsumer.java:156)
[info]   at com.ibm.msg.client.wmq.internal.WMQConsumerShadow.initialize(WMQConsumerShadow.java:1173)
[info]   at com.ibm.msg.client.wmq.internal.WMQSyncConsumerShadow.initialize(WMQSyncConsumerShadow.java:128)
[info]   at com.ibm.msg.client.wmq.internal.WMQMessageConsumer.<init>(WMQMessageConsumer.java:486)
[info]   at com.ibm.msg.client.wmq.internal.WMQSession.createConsumer(WMQSession.java:929)
[info]   at com.ibm.msg.client.jms.internal.JmsSessionImpl.createConsumer(JmsSessionImpl.java:1012)
[info]   at com.ibm.msg.client.jms.internal.JmsSessionImpl.createConsumer(JmsSessionImpl.java:1097)
[info]   at com.ibm.msg.client.jms.internal.JmsContextImpl$SessionWrapper.createConsumer(JmsContextImpl.java:1989)
[info]   ...

superseded by https://github.com/fp-in-bo/jms4s/pull/377


vJoy injector support

Work in progress. Resolves #39, and takes inspiration from this project.

I'm very excited about this project. I've been waiting to try it for days.


azure/governance-regional: add optional resource group lock

Fixes #135

This will introduce a breaking change. Everyone will have to add the parameter lock_resource_group to all objects in the resource_group_configs list.
This won't be needed in the future with: https://github.com/XenitAB/terraform-modules/issues/140

@XenitAB/team-devops breaking change


Issue Installing MariaDB on CentOS7 WSL: Failed to get D-Bus connection: Operation not permitted

First of all, awesome work on creating this for WSL. Really really useful, thank you.
I am following instructions given in here to install MariaDB on CentOS7 for WSL in Windows 11.
PS C:\Users\joe> wsl -l -v
  NAME                   STATE           VERSION
* Ubuntu-22.04           Stopped         2
  docker-desktop         Running         2
  docker-desktop-data    Stopped         2
  CentOS7                Stopped         2

sudo yum install mariadb-server works with no problems as expected. However, sudo systemctl start mariadb will run into the following issue:
[root@mycomputer etc]# systemctl start mariadb
Failed to get D-Bus connection: Operation not permitted

So I enabled systemd by adding the following to /etc/wsl.conf
[boot]
systemd=true

I shutdown wsl and restarted it. systemctl list-units --type=service will produce the proper results indicating systemd is working. However, I am still getting Failed to get D-Bus connection.
Is there any way to get MariaDB server to work in CentOS 7 for WSL?

在wsl中执行systemctl相关命令报错 Failed to get D-Bus connection: Operation not permitted
备份systemctl
mv /usr/bin/systemctl /usr/bin/systemctl.bak
写入systemctl脚本
curl https://gitee.com/umico/docker-systemctl-replacement/raw/master/files/docker/systemctl.py > /usr/bin/systemctl
授予执行权限
chmod +x /usr/bin/systemctl

@fzxfzx12345 升级wsl软件本身即可支持识别/etc/wsl.conf文件

For additional info on using systemd under WSL, please see https://github.com/mishamosher/CentOS-WSL/issues/7


feat: snapshots

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Exclude test on windows

Test failed on win32 pass on win64. JBS issue fixed though the failure is still there.
Signed-off-by: Sophia Guo<EMAIL_ADDRESS>

No win32 with openj9. Openj9 win64 failed for different issue and has been excluded with specific issue.


Does webrtc support H.265 ?

Hello, I would like to know if webrtc supports H.265 ?
If not, will you support it recently ?
Thanks a lot!

This library does not natively support H265 at the moment, and I don't know of any upstream plans from google to add the encoder/decoders for it.
A VideoEncoder/DecoderFactory can be passed in to add support for it, but that'll require implementing the codec yourself. There's some information here: https://groups.google.com/g/discuss-webrtc/c/6kokPpnSO3Y

This library does not natively support H265 at the moment, and I don't know of any upstream plans from google to add the encoder/decoders for it.
A VideoEncoder/DecoderFactory can be passed in to add support for it, but that'll require implementing the codec yourself. There's some information here: https://groups.google.com/g/discuss-webrtc/c/6kokPpnSO3Y

Thanks a lot.


You can use the qcow2 format for the disk image file?

hi kroese:
You can use the qcow2 format for the disk image file. If it is possible, how can the parameters be adjusted?

hi,kroese
What environment variables do I need to set in order to run a Windows qcow2 disk image file that I have already created?
Certainly! Here is a detailed example.

hi,kroese
What environment variables do I need to set in order to run a Windows qcow2 disk image file that I have already created?
Please provide a detailed example.

The progress bar keeps loading. After waiting for a long time, a blue screen appears. I don’t know how to solve it.

Regardless of whether I use the qcow2 format or not, I encounter the same installation problem with Windows 7, Windows 10, and Windows 2008.

Once I eliminated the qcow2 disk format and set the environment variable KVM to N, the installation succeeded. But upon accessing the system, I encountered some information—can this be altered to reflect the host machine's details?

Is it possible to customize or alter the yellow highlighted section in the image to match the host machine's CPU information?

thinks

thanks

I'm considering testing with Ubuntu 22.04; which operating system do you typically use for your tests?

I'll run some additional tests here and will provide you with feedback once I have the outcomes.

hi kroese :
I've encountered a new issue where I replaced the data.qcow2 file with my own qcow2 file and renamed it to data.qcow2. However, the system within my qcow2 file does not have the virtio drivers installed (neither for the hard disk nor the network card). This led to a system crash with a blue screen when I tried to start it using the docker-compose file. Could you tell me how to alter the disk and network card drivers in the qemu command to use the compatible ide and e1000 drivers instead? I would appreciate any solution you might suggest, thank you.

My current qcow2 file contains programs that have been running for many years (and are no longer maintained) along with a large amount of data. I'm hesitant to install virtio drivers for fear of encountering issues. Now, my only option is to modify the disk.sh and network.sh scripts within the qemu-docker project, but I still encounter a blue screen after making modifications. I think the problem still lies in how I modified the disk.sh script.Here is the content of the modified scripts.
[root@localhost ~]# vim /root/qemu-docker-master/src/network.sh

Could you advise which function's content within the disk.sh] script needs to be modified? I would appreciate your guidance, thank you.


Add to Seen History Date is One Day Behind Selection

Hello,
I am accessing MediaTracker 0.1.0-beta16 on my Android phone. When I go to add a movie to my watch history, the date/time pickers are initially set to the correct current date and time. However, if I go to select a prior date, the date it actually sets in the box is always one day earlier. For example, I selected 2022-05-20, but the box changed to 2022-05-19 instead. In order to get the box to the correct date, I had to pick the day after the one I actually wanted.
Thanks.

It's similar bug like in #261, I missed this one place, it should be new fixed in version 0.1.0-beta.17

Tested the new release, that worked for me, thank you!


.28 throws errors on the actual docker image

The actual Version is not working on the home bridge docker image. Can not provide more details at the moment. Manual install of homebridge lib helps, but gives other yellow warnings.
https://forum.smartapfel.de/forum/thread/306-homebridge-hue/?pageNo=13

Not sure which warnings you're referring to:

NodeJS version: I develop and test agains the latest LTS version of NodeJS.  Other versions usually work, but we've seen some incompatibilities in the past.  Use these at your own risk;
I don't use docker; I know of it, but cannot help you with the setup;
With recent firmware, the Hue bridge no longer supports software interaction with the link button.  This functionality has been removed from homebridge-hue, including the corresponding linkButton config.json key;
heartbeat drift warning (from homebride-lib): I found that, depending on NodeJS load, the heartbeat isn't happening every second exactly, and drifts over time (so there's less than 86400 beats per day).  homebridge-lib tries to compensate, by taking into account the drift when scheduling each next heartbeat.  It issues a warning when the delta between scheduled time and actual time is greater than 0.25 seconds (250 milliseconds).  Some drifting during startup of homebridge is to be expected, but values of almost half a minute shouldn't occur.  There's probably another plugin doing synchronous I/O, preventing the heartbeat from happening.
There's no need to install homebridge-hue nor homebridge-lib with --unsafe-perm as they don't contain any C / C++ modules that need to be compiled.
I sometimes get the EACCESS error on ~/.npm as well.  Deleting ~/.npm solves it.

Some more backgound info (I read German, but I don't write it well):

I'm (slowly) preparing for dynamic platform accessories (which get persisted between homebridge restarts).  This will be based on the generic logic in homebridge-lib.
I setup homebridge-lib as a peer dependency so a single instance of homebridge-lib can be shared between multiple plugins, very much like homebridge itself.  In particular, using a single heartbeat schedular including a single flush of ~/.homebride/accessories/cachedAccessories.
homebridge-lib is now a (fake) homebridge plugin, so it can be updated from homebridge-config-ui-x.  It doesn't expose anything, and doesn't need to be included in config.json.  It does need to be loaded however, for plugins that make use of the dynamic accessores (currently homebridge-ws and homebridge-otgw), so make sure to include it when you specify plugins in config.json.
I'm trying to simplify the setup, homebridge-hue-utils is now integrated in homebridge-hue.  Homebridge-hue now creates bin symlinks only for the commands it includes (ph and dc_eventlog); the symlinks for hap, json, and upnp are maintained by homebridge-lib.  npm might give some warnings when un/re-installing and/or leave some stale symlinks.

Thanks for your detailed explanation.  Cleaned up and now the only error I get is the
[homebridge-lib.Lib] warning: heartbeat 1, drift 309
The issue can be closed. Thanks again!


fix calcFlightTime and calcFuel

Codecov Report

:exclamation: No coverage uploaded for pull request base (develop@947fdb9). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff             @@
##             develop      #36   +/-   ##
==========================================
  Coverage           ?   41.64%
==========================================
  Files              ?       98
  Lines              ?     7129
  Branches           ?        0
==========================================
  Hits               ?     2969
  Misses             ?     4094
  Partials           ?       66

Impacted Files
Coverage Δ

ogame.go
5.83% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 947fdb9...f154d25. Read the comment docs.


Sonic the K-Day Hedgehog: Watch +919987686385

Sonic the K-Day Hedgehog requests the watch of the watch_number +919987686385. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Approved by tripleee in Charcoal HQ


viv: insn: string: handle viv bug around substrings

closes #1271
try to detect the case in which viv incorrectly returned a substring length and truncate the returned data appropriately. i think this will work better for ASCII than for UTF16 because we can better guess about embedded NULL bytes.

Checklist

[ ] No CHANGELOG update needed

[ ] No new tests needed

[ ] No documentation update needed

ca9c940 closes #1278


DirectStream Input (OBS Virtual Webcam)?

Feature request

What problem does this feature solve?
Looking for a way to feed Hyperion with OBS data I finally ended up with USB Grabber again.
I also talked to the developers of Hyperion but they don't seem to be interested in optimizing the software for PC gaming.
Anyway, I found this project here by chance and was enthusiastic from the beginning. Interesting I found the possibility of ScreenRecording using microsoft media foundation.
Finally it is possible purely by software to enjoy my Ambilight with Doom (Vulkan)!!! No other grabber could offer me this up to here! Yes I know, it is not perfect ... but therefore also the request here to work on integrating OBS.
What does the proposed API look like?
How should this be implemented in your opinion?
One possibility would be for example with OBS Virtual Webcam (DirectStream). But you will with certainty
Are you willing to work on this yourself?
I would, but I can't. That is a little too high for me

Hi
OBS under Windows doesn't expose media foundation interface and it uses DirectShow instead. I used some other software that can create such interface using media foundation (for example to stream a video from mobile) so it's definitely possible but it seems OBS lags here. I know there is an issue on macOS with OBS: it's probably related to some missing video capture mode in HyperHDR but can't say more without provided logs.
I don't have a plan to implement any DirectX/Stream related capturing under Windows because it's legacy solution compared to Microsoft media foundation. But I do have a plan to create a plunging for OBS to send a stream to HyperHDR. Probably two major version away as a lot is going on in the HyperHDR developing right now and we lack of resource at the moment.

Hi
OBS under Windows doesn't expose media foundation interface and it uses DirectShow instead. I used some other software that can create such interface using media foundation (for example to stream a video from mobile) so it's definitely possible but it seems OBS lags here. I know there is an issue on macOS with OBS: it's probably related to some missing video capture mode in HyperHDR but can't say more without provided logs.
I don't have a plan to implement any DirectX/Stream related capturing under Windows because it's legacy solution compared to Microsoft media foundation. But I do have a plan to create a plugin for OBS to send a stream to HyperHDR. Probably two major version away as a lot is going on in the HyperHDR developing right now and we lack of resource at the moment.

Hi,
thank you for the answer and I am very happy to hear that you are working on a plugin for OBS !
For now I'll make do with OBS+ManyCam. That works quite well for now.
Nevertheless, I have to ask a question that doesn't quite fit here. I do not understand the adjustment for HDR with the LUT file. Did I miss something or is there really no good tutorial for this?
Do you have a forum where you can discuss something like that ?

Almost all the users were using the default LUT table (previously you could only tweak the brightness) so there was little demand for better tutorial ;) There is some info in the manifest of the latest release as it brought new features related to the LUT table subject. If something is not clear you can ask here of course.

Hi,
thanks for offering to help me here.
I do not quite understand the LUT yet. As soon as I turn on HDR in Windows, everything appears white! Can you possibly help me here?

Assuming that you are using the latest version  most probably the video stream to the grabber is not in fact HDR and it's captured by the grabber as SDR (SDR capturing works OK for almost all modern grabbers). In that case you don't need hdr tone mapping  because it leads to picture overexposed. You should take a look at the video preview before turning on HDR tone mapping in the HyperHDR and compare it to the examples on the main github hyperhdr page, how it should look before (tone mapping off) with colors washed-out and after (tune mapping on).

Then I don't quite understand how I can do HDR and still not have washed out colors?

As I wrote, probably you dont have a hdr signal on the grabber input or its for example Ezcap 320 that can do partial tone mapping and it needs a special LUT table. Take a screenshot similar as above with turned OFF hdr tone mapping in hyperhdr assuming that you turned ON hdr in Windows 10. Then we will see if you need it.

As I wrote, probably you dont have a hdr signal on the grabber input or its for example Ezcap 320 that can do partial tone mapping and it needs a special LUT table. Take a screenshot similar as above with turned OFF hdr tone mapping in hyperhdr assuming that you turned ON hdr in Windows 10. Then we will see if you need it.

Hi,
I have attached a screenshot.

by the way:
I have changed the settings in the meantime. I got a cheap HDMI grabber for 5€ at Amazon (https://tinyurl.com/2nxm37tu). It is not the fastest, but for now it is okay for me. Until your OBS plugin comes I'm happy with it for now.
With OBS I make a full screen project on the HDMI grabber and get me from there the image via HyperHDR.
I now have one less software...

It's seems that the HyperHDR HDR tone mapping is still ON.
Please set if OFF  and then take a screenshot.
BTW it's classic example of unnecessary using of Hyperhdr HDR tone mapping on SDR source again.

by the way:
I have changed the settings in the meantime. I got a cheap HDMI grabber for 5€ at Amazon (https://tinyurl.com/2nxm37tu). > It is not the fastest, but for now it is okay for me. Until your OBS plugin comes I'm happy with it for now.

Probably it's MS2109 - many of our users use it with good results.

It's seems that the HyperHDR HDR tone mapping is still ON.
Please set if OFF and then take a screenshot.
BTW it's classic example of unnecessary using of Hyperhdr HDR tone mapping on SDR source again.

by the way:
I have changed the settings in the meantime. I got a cheap HDMI grabber for 5€ at Amazon (https://tinyurl.com/2nxm37tu). > It is not the fastest, but for now it is okay for me. Until your OBS plugin comes I'm happy with it for now.

Probably it's MS2109 - many of our users use it with good results.

Tone mapping is off... this comes from OBS... :(

From what I found OBS doesn't support system HDR unfortunately. Maybe that it manifests?

I'm a little confused myself.
I remember some time ago I tried to record my HDR screen with OBS.
And here I had a rather dark image with washed out colors. Actually exactly what we expect here ?
I don't know why I get this bright image instead.
Do you have an alternative idea how I can enjoy HyperHDR with Windows 10 and HDR ?

Probably they are working to implement it. But it seems OBS captured already SDR stream and then they apply tone mapping unnecessary. Try to downgrade OBS a year or so (uninstall also that virtual camera first)? Other solution is to use USB grabber.

I'm going to try this with a downgrade.
Yes but USB Grabber looks at Displayport with HDR and 170hz, freesync etc probably rather bad or?
I think here it goes from the frying pan into the fire....

Dont know of any grabber works with gsync or freesync.
170hz propably is also too much for MS2109.
Only Ezcap 320 supports quite high framerate: 2160p60, 1440p144, 1080p240. At least on the paper.

the MS2109 is actually okay for now.
I had hoped that I could simply duplicate the image.
I wanted to run the grabber at 720p/60hz and my monitor at 1440p/170hz and then just clone.
But that doesn't really seem possible with Windows/Nvidia like that. This is possible but the Windows UI lags terribly...
Here is a video of my current method which only works without HDR:
OBS Projection to HDMI Grabber -> HyperHDR.
https://user-images.githubusercontent.com/4953452/121016289-3bfa0a00-c79c-11eb-8622-66bd317ebe7e.mp4
You can see that it is slightly delayed but it's totally okay for me
I'll just have to keep trying....

Try to disable smoothing because with wrong settings (not corresponding to the led driver capability) it could affects performance. Black bar border detection is nonessential also.
Try lower resolution in OBS' camera and especially lowering framerate is a must (from the list or use frame rate decimation option). Force different video encodings if available. Probably that virtual camera has to make a color space conversion before streaming and then we have to make another conversion in HyperHDR: some encodings could work better in that situation.

Hi,
I wanted to let you know something. I had a look at the Windows 11 Preview today and found out that it is possible to mirror the 165hz screen with a USB grabber display without any jerky effects on the 165hz screen.
It seems that Microsoft finally fixed this annoying bug with Wddm 3.0 !
I'll have to try the whole thing with a virtual display...

Many changed from this time, and HyperHDR currently offers software screen grabbers for every OS. If a new interesting functionality would appear in OBS, I would again re-consider creating a plugin. I am closing the issue for now.


Intuit.Ipp.Data.dll File Version

In v7.5 of this SDK Intuit.Ipp.Data.dll was versioned like the other dlls with <IP_ADDRESS>, but now in the latest releases it is <IP_ADDRESS>. I assume this is unintentional? I assume it should be versioned to match the other Intuit.Ipp dlls or at least be higher than its prior version. I made a workaround to get this to deploy with our internal updater utility so it saw the file as newer than the <IP_ADDRESS> version.
Currently:

Back in v7.5 of this SDK:

I meant to post this to QuickBooks-V3-DotNET-SDK...


Pigeon: unable to spawn isolate

Unhandled exception:
IsolateSpawnException: Unable to spawn isolate: Error: Could not resolve the package 'pigeon' in 'package:pigeon/pigeon_lib.dart'.
/var/folders/xt/mwq_g0y15x5dqzsdc3zym1_r0000gn/T/27uiPu/_pigeon_temp_.dart:4:8: Error: Not found: 'package:pigeon/pigeon_lib.dart'
import 'package:pigeon/pigeon_lib.dart';
       ^
/var/folders/xt/mwq_g0y15x5dqzsdc3zym1_r0000gn/T/27uiPu/_pigeon_temp_.dart:7:23: Error: Getter not found: 'Pigeon'.
  sendPort.send(await Pigeon.run(args));

when flutter pub run pigeon on 1.17
From GDS China video comment https://www.bilibili.com/video/BV11z4y1y7ph

I'm unable to reproduce this problem, I'd need to have some reproduction steps.  Is the user potentially running from global activate instead of using pigeon as a dev_dependency?
However they produced it, I suspect it will be eliminated with the migration to the Dart analyzer here: https://github.com/flutter/packages/pull/381

Since we don't have repro steps, and the migration to analyzer landed, I'm going to close this as there doesn't seem to be anything else actionable at this point.


how to release gpu memory in the garbage pool?

As I can see,  there're a lot of issues about gpu memory leaking, and always end with something like:

Memory isn't really 'leaked'. It's released to a pool. The pool will only be emptied when an allocation returns out of memory.

However, when I use multiprocessing,  it is really painful,  because the "pool" isn't share between process. E.g. I have 4 processes, one ocuupied 11g, when another process get "out of memory", it cannot release the first process's garbage pool. And if I do not call the first process forever, the garbage will be there forever. Is there any method to release memory manually? Or can I set a memory limitation for a process?

@weishy-xun - did the suggestion above to use MXNET_GPU_MEM_POOL_RESERVE help with the issue?
Since there was no response in over a month I think it is best to close the issue, feel free to re-open if you think it is done in error.
@sandeep-krishnamurthy kindly close the issue and label as "CUDA" (asking for a new label)


[Patterns] different errors in case of non-exhaustive switch

The following code produce different errors
int test1(Object obj) { // COMPILE_TIME_ERROR.BODY_MIGHT_COMPLETE_NORMALLY
                        // The body might complete normally, causing 'null' to be returned, but the return type, 'int', is a potentially non-nullable type.
  switch (obj) {
    case int(isEven: true) as int:
      return 1;
    case int _:
      return 2;
  }
}

int test2(Object obj) => switch (obj) { // COMPILE_TIME_ERROR.NON_EXHAUSTIVE_SWITCH
                                        // The type 'Object' is not exhaustively matched by the switch cases.
  int(isEven: true) as int => 1,
  int _ => 2
};

String test3(bool b) {
  switch (b) {  // COMPILE_TIME_ERROR.NON_EXHAUSTIVE_SWITCH
                // The type 'bool' is not exhaustively matched by the switch cases.
    case true:
      return "true";
    case false when b == false:
      return "false";
  }
}

main() {
  test1(1);
  test2(2);
  test3(true);
}

Why in the first case we have an error that differs from cases 2 and 3?
Tested on the Edge SDK on Linux (March 28, 2023)

I think the error in test1 is OK, cf. this in-depth explanation and this issue summary:
The static type of obj is Object at the beginning of the switch statement. The fact that as int did not throw promotes the matched value type to int in that case, but not in subsequent cases. So obj may still be a String or something else according to the exhaustiveness analysis when we consider case int _:, and the switch statement will then complete normally, and so will the function body. It is not an error for the switch statement to complete, but it is an error for the function body to return null, which is the semantics of reaching the end of the body.
In test2 and test3 it is an error for the switch to be non-exhaustive, because it is a switch expression.


[multimodal]Enable backbone freezing to boost finetuning speed and save GPU usage

The current two-stage/layerwise-decay learning rate settings supports use different lr in head and non-head layers. It also allows user to set lr in non-head layers to 0. However,

Most models are more complicated than backbone + head. And we may want to use two stage on head v.s. non-head, and layer freeze on backbone v.s. non-backbone. (e.g. backbone_lr = 1e-5, neck_lr=1e-5, head_lr=1e-3 and backbone_lr = 0, neck_lr=1e-4, head_lr=1e-4 may both converge fast and nice but backbone_lr = 0, neck_lr=0, head_lr=1e-4 may not be a good one)
Only setting lr = 0 in two-stage does not save GPU memory. Setting require_grad=False for backbone parameters can save lots of GPU memory and enable us to run a larger batch_size or a larger model.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thank @FANGAreNotGnu, can you run some quick tests to make sure the memory footprint has been reduced post this change?

Thank @FANGAreNotGnu, can you run some quick tests to make sure the memory footprint has been reduced post this change?

Yes I did, the YOLOX-L GPU usage with batch_size=8 is reduced from >16GB to <10GB. (trainable parameters from 54M to 27M)

The current two-stage/layerwise-decay learning rate settings supports use different lr in head and non-head layers. It also allows user to set lr in non-head layers to 0. However,

Most models are more complicated than backbone + head. And we may want to use two stage on head v.s. non-head, and layer freeze on backbone v.s. non-backbone. (e.g. backbone_lr = 1e-5, neck_lr=1e-5, head_lr=1e-3 and backbone_lr = 0, neck_lr=1e-4, head_lr=1e-4 may both converge fast and nice but backbone_lr = 0, neck_lr=0, head_lr=1e-4 may not be a good one)
Only setting lr = 0 in two-stage does not save GPU memory. Setting require_grad=False for backbone parameters can save lots of GPU memory and enable us to run a larger batch_size or a larger model.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Looks like this PR only addresses the second point, i.e., setting the requires_grad=True for backbone parameters.

Changed the design based on some offline discussions.


The number of started jobs is a lot bigger than the number of workers

We currently have nearly 2000 started jobs, and "only" 200 workers

My guess: the jobs raise an uncaught exception and are never finished properly in the database. They will be deleted afterwards anyway (zombie killer) but possibly we must fix something here.

Note that the three affected steps are dataset-opt-in-out-urls-count, config-opt-in-out-urls-count, split-opt-in-out-urls-count, which at least seems coherent in some way.
maybe @AndreaFrancis you want to check this?

Sure, I will check

Interestingly, there are jobs with "started" status but have a  "finished_at" value and a cache record in the DB. I think only the status is not being set correctly, will put here my findings.

I found some other records in the queue for other steps (Different than dataset-opt-in-out-urls-count, config-opt-in-out-urls-count, split-opt-in-out-urls-count) using filter {status:"started", finished_at:{$exists:true}}. But I was not able to reproduce the issue.
I think setting the TTL index to drop only for those status success/error would help us identify the problem.

Hmmm. No need to investigate too much. If we don't find the root reason quickly, we can let it unchanged: anyway, the cron backfill will fix the remaining issues once a day

Hmmm. No need to investigate too much. If we don't find the root reason quickly, we can let it unchanged: anyway, the cron backfill will fix the remaining issues once a day

Sure, anyway I think we should not delete the inconsistent records, those are not being finished by zombie killer, and looks like queue's finish method is not letting us know if there is a wrong combination of state+dates https://github.com/huggingface/datasets-server/blob/main/libs/libcommon/src/libcommon/queue.py#LL632C15-L632C15 it returns a boolean but nowhere is being evaluated.
I opened https://github.com/huggingface/datasets-server/pull/1325, and would like to understand what is happening and if there is a pattern for this behavior.

Note that since we change the TTL index, we now have "waiting" jobs with a non-null finished_at field, which shows up in the metrics. Maybe we should change the way we compute the metrics, if this issue is not solved soon


read_b2_output( b2fgmtry ) includes timesa key, which doesn't belong in this file

In test/runtests.jl:
contents = read_b2_output("$(@__DIR__)/../samples/b2fgmtry")
returns a structure that contains "timesa" within "data", but "timesa" is a b2time.nc thing; it shouldn't be found in b2fgmtry

That was intentional on my end. Edge_profiles.grid_ggd is supposed to be an array of structures corresponding to a time evolution. This time is stored in edge_profiles.grid_ggd[:].time . For consistency with any future implementation of time dependent grids, I introduced a timestamp 0.0 for the grid while storing it from b2fgmtry. Let me know if you still think we should remove it.

That might be overdoing it on future compatibility, but okay, we can leave it for now if you think it's a good idea.


2.0: Spacing between bars with percentages set to 1

The border width of the dataset is set to 0.
In options:
        scales: {
          xAxes: [{
            barPercentage: 1.0,
            categoryPercentage: 1.0,
            ...
          }]
        }

There's still thin lines randomly throughout the chart.
This is more noticeable as the number of entries increases (this chart normally has 1024 values).
I've also tried setting stacked: true on both axes, but this doesn't seem to have an impact.
Is there any work around to eliminate this spacing?

@cpboyd I think this might be either

a rounding issue when determining the pixel values
Something to do with the border color.

Could you drop this in a JSFiddle?

@etimberg https://jsfiddle.net/33m4cf21/
I noticed that removing the Y-axis label reduced the amount of spaces drastically.
So it seems that the Y-axis label has a big impact in causing this issue... Although, it still happens.

Actually... I'm not sure that the Y-axis label has much to do with it, after all.
I noticed that it's more about the width of the canvas, which does seem to hint at a possible rounding issue.
If you resize the window, the lines will increase/decrease in a periodic fashion ranging from zero lines up to a line between each bar and then back to zero lines.

@cpboyd it looks like your fiddle is for a radar chart. Did you paste the wrong link?
I don't think we do a lot of rounding of pixel values. There are none in the Chart.elements.Rectangle object.

@etimberg
I'm not quite sure what happened there.
This should be correct, though: https://jsfiddle.net/ewqadp6t/

I've narrowed this down to the tickWidth calculation within getRuler:
var tickWidth = (function() {
    var min = xScale.getPixelForTick(1) - xScale.getPixelForTick(0);
    for (var i = 2; i < this.getDataset().data.length; i++) {
        min = Math.min(xScale.getPixelForTick(i) - xScale.getPixelForTick(i - 1), min);
    }
    return min;
}).call(this);

The issue is that it finds the smallest tickWidth and applies it uniformly, without changing the xScale at all.
Ideally, it seems like it would take this tickWidth and then set the chart's width to tickWidth* datasetCount and then rescale the tickLeft calculations accordingly.

@cpboyd Great find! I think it can be updated to support variable tick width. That would make the most sense to me. It could be a little weird on time scales, but we can cross that bridge when we come to it.

@cpboyd I made some progress on this tonight. I still need to make sure I haven't broken anything for bars on time scales and do the same fix for the horizontal bar
Before

After

This bug is back in version version: 2.7.2 -  text on the chart is crisp, so it's not a canvas issue, but it's like the bars are using "pixels" which are larger than the pixels on my screen (much, much larger if I zoom in).  Anyone have a workaround?


1.8.4 release

Target date: 2018-12-10.

[x] close tagged issues
[x] check version and change log
[x] tag
[ ] build and uploads dists

Share and enjoy!


dragHitSlop property wrong data type

Describe the bug
dragHitSlop is an object not a number
Additional context
Add any other context about the problem here.

Hello there, you created a nice module here! :+1:
VS Code is just complaining about wrong type that I pass to the dragHitSlop property.

thanks! fixed in 2.5.1

That was quick! Thank you!


ResourceOwnerPasswordTokenGranter handles UsernameNotFoundException

Fixes gh-1408

@aschatten Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@aschatten Thank you for signing the Contributor License Agreement!

Thanks for the PR @aschatten. This is now in master.


ModuleNotFoundError: No module named 'click' [16116] Failed to execute script main

Hi,
Apologies if I'm going about everything the wrong way - this isn't something I've ever done before (both using a program like this really and trying to get help/ask about a problem on Github).
Trying to use the handy pre-compiled Windows binary version of your program and then presumably operate it via command line/powershell, however whenever I try to use it, be that to actually do something with a ROM or even execute the --help command as a test, I get the following error:
Traceback (most recent call last):
  File "luvdis\__main__.py", line 3, in <module>
ModuleNotFoundError: No module named 'click'
[16116] Failed to execute script __main__

Not sure what the issue is or what I'm doing wrong unfortunately. I do have Python installed on my system, though I'm not sure what else it could be. Thanks.

Sorry for lateness; I'll look into this, maybe I just forgot to embed the dependency?
If you already have python installed, I'd recommend just installing via PiP:
python3 -m pip install luvdis

(Probably from an admin Powershell or similar, or you could try adding --user to that command)


Form completed screen

Can we improve this view:

[ ] Add "Create new proposal" button
[x] Add link to the post on forum
[x] Redirect user to the forum post in 5 seconds. Show a message and a timer for this.
[ ] Stick the footer to the bottom

@tomislavmamic what you mean when you wrote: "Add "Create new proposal" button" ?
As we added project proposal to locked category, this two tasks: "Add link to the post on forum" and "Redirect user to the forum post in 5 seconds. Show a message and a timer for this." have been solved.


Add optional instance pre-provisioning

In order to have instances at the ready at all times and
thus speed up CI runs because instance provisioning is done
upfront, add a mechanism and the related parameter to configure
a minimum number of instances that must be ready at all times.
This change is backward-compatible: if the new parameter 'instancesMin'
is set to zero (its default value), no pre-provisioning happens.
By combining a non-zero value of 'instancesMin' and a zero retention
time, one can favour the pre-provisioning of new instances over the
reuse of slaves.
Drive-by: fix test whose failure wasn't checked.

@olivergondza Thanks for the quick and thorough review!  I believe I've addressed all your comments, please have another look when you get a chance.

@olivergondza Thanks again for the detailed review; I've implemented most of your suggestions (see 3rd commit), short of the one suggestion I disagree with, but I've provided an explanation about why I believe it wasn't a good idea.

@rabadin, thanks a lot! Merging and releasing right away to get some feedback ASAP.


Cannot read properties of undefined (reading 'split')

Hi,
I'm trying to parse junit xml reports which were prepared by gtest framework using java-junit format and it's working well for report files which contains successful tests only.
While reporter is trying to parse xml with failed tests - I'm facing Error: Cannot read properties of undefined (reading 'split')
Sometimes our tests are not handling exceptions correctly (because this tests are coming from upstream project) and test case can miss failure message.
  <testsuite name="SpellingMenuObserverTest" tests="1" failures="1" disabled="0" errors="0" time="34.336" timestamp="2022-05-24T09:57:08">
    <testcase name="InitMenuWithCorrectWord" status="run" time="34.336" timestamp="2022-05-24T09:56:34" classname="SpellingMenuObserverTest">
      <failure message="" type=""></failure>
    </testcase>
  </testsuite>

May I ask you to change current behaviour of parsing such messages if it's missing details. I've tried multiple command line parsers and they are working this way, just to check an assumption that gtest junit xml reports are really compatible with junit parsers.
Thanks in advance!

@ironstas1k , this looks very similar to a bug we've already fixed:
https://github.com/phoenix-actions/test-reporting/issues/12
Can you try the latest version and see if that works for you?

Closing for now, but if we need to re-open it again, we can 👍


Fetching suppression file through Git

Is your feature request related to a problem? Please describe.
In a setting with multiple microservices, maintaining a suppression file in each service is getting quite tiresome.
Tried using both raw url and git url as suppressionFile, but the first ends up with 404, and the second is handled as a local file path.
Describe the solution you'd like
Fetching suppression file from git/bitbucket.
suppressionFile = 'git@bitbucket.org:repo.git/path-to-file.xml'

Describe alternatives you've considered
Adding it as a package seems like an extra step for very little gain

You should be able to use a http(s) url to the raw suppressionfile (but need to ensure that it is either publicly visible, or your git raw URL includes credentials). My gut feel is that your HTTP 404 stems from the suppression-file not being publicly visible.
Supporting git URLs would mean we have to embed a git client as a dependency.


Make fact args use the {{ format }}ing that operation args do

Closed @ https://github.com/Fizzadar/pyinfra/commit/de1b8bde324d0870791b34f09e2ecff3b41f71f8


Embed should support autoplay functionality

For example for the columns block on https://main--helix-sportsmagazine--headwirecom.hlx.page/ or the embed block in the article here https://main--helix-sportsmagazine--headwirecom.hlx.page/content-v2/golf-news-tours/_default/article/2023/1/tom-kim-dunks-approach-shot-asks-fantastic-question-sentry-toc

I propose to check if the block has the classname “autoplay” to toggle the functionality


Spruce cone model error

1.12.2 forge 2860  DynamicTrees-1.12.2-0.9.29

The model file hasn't changed recently and I notice you have a resource pack enabled. It is most likely something to do with the resource pack, please try disabling it and see if the issue persists. If it doesn't please tell me the name of the resource pack and I'll try and see what's causing it.

模型文件最近没有更改，我注意到您启用了资源包。这很可能与资源包有关，请尝试禁用它，看看问题是否仍然存在。如果不是，请告诉我资源包的名称，我会尝试看看是什么原因造成的。

It has been found that it is a resource pack problem. Thank you


unsupported value type

Host operating system: output of uname -a: Linux 4.18.0-477.15.1.el8_8.x86_64 #1 SMP Fri Jun 2 08:27:19 EDT 2023 x86_64 x86_64 x86_64 GNU/Linux
flexlm_exporter version: output of flexlm_exporter --version
flexlm_exporter, version  (branch: non-git, revision: non-git)
  build user:       myuser@918d31421ff1
  build date:       20231002-16:54:34
  go version:       go1.20.7
  platform:         linux/amd64
  tags:             netgo static_build

Current master copy built from source
flexlm_exporter command line flags and relevant YAML configuration file information

--path.config="/opt/flexlm_exporter/licenses.yml" \
    --path.lmutil="/opt/flexlm_exporter/lmutil" \
    --log.level=debug

Are you running flexlm_exporter in Docker?
No
What did you do that produced an error?
start flexlm daemon
What did you expect to see?
metrics from license servers
What did you see instead?
the /metrics endpoint only shows the stock Go mem statistics. No info from license servers
Oct 02 18:55:16 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:55:16.183Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:53:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:53:44.203Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:48:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:48:44.203Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:43:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:43:44.202Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:38:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:38:44.202Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:33:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:33:44.203Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:28:45 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:28:45.394Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:28:44 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:28:44.206Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:26:02 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:26:02.906Z caller=flexlm_exporter.go:89 level=debug msg="collect query:" filters="unsupported value type"
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.625Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=<IP_ADDRESS>:9319
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.625Z caller=tls_config.go:274 level=info msg="Listening on" address=<IP_ADDRESS>:9319
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=config.go:49 level=info -=/opt/flexlm_exporter/licenses.yml
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=config.go:48 level=info msg="Loading license config file:"
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=flexlm_exporter.go:123 level=info msg="Enabled collectors"
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=flexlm_exporter.go:202 level=warn msg="FLEXlm Exporter is running as root user. This exporter is designed to run as unprivileged user, root is not required."
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=flexlm_exporter.go:200 level=info msg="Build context" build_context="(go=go1.20.7, platform=linux/amd64, user=myuser@918d31421ff1, date=20231002-16:54:34, tags=netgo static_build)"
Oct 02 18:25:28 example.server.net flexlm_exporter[2116849]: ts=2023-10-02T18:25:28.616Z caller=flexlm_exporter.go:199 level=info msg="Starting flexlm_exporter" version="(version=, branch=non-git, revision=non-git)"
Oct 02 18:25:28 example.server.net systemd[1]: Started flexlm_exporter for Prometheus.

Thanks to the bug fix for #77 I can start the flexlm daemon.
However, now, there appears to be issue getting the actual information to pull. The licenses.yml is the same one I used in previous versions without issue.
licenses:
  - name: server1
    license_server: 1055@server1
    monitor_users: True
    monitor_reservations: True
  - name: server2
    license_server: 27000@server2
    monitor_users: True
    monitor_reservations: True

It appears the changes between versions is not liking the data retrieved from lmutil.
This is the version of lmutil:

[~]# ./lmutil lmstat -v
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
lmstat v<IP_ADDRESS> build 242433 x64_lsb

Is this an issue where compatibility is broken with my version of lmutil?

@cloudrck thanks for reporting. Indeed it could be related to your lmutil output.
unsupported value type is not related to your issue, but more a logging library problem.
If you could check the lmutil output against the format of the fixtures here, which are the one this exporter is tested against, you should note the differences. Let me know if you find a different format there.

Thanks for the reply and info.
Here is the output of about 3 license servers.
[~ flexlm_exporter]# ./lmutil lmstat -c 4100@l-server01 -a
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Flexible License Manager status on Mon 10/2/2023 18:32

License server status: 4100@license-server01
    License file(s) on license-server01: C:\Magicdraw_license_manager\license.lic:

license-server01: license server UP (MASTER) v11.19.0

Vendor daemon status (on license-server01):

     cameo: UP v11.19.0
Feature usage info:

Users of CC888888Connections:  (Total of 2 licenses issued;  Total of 2 licenses in use)

  "CC888888Connections" v2022, vendor: cameo, expiry: 02-jun-2025
  vendor_string: product=CC4TWC|profile=0|edition=10\u0020Connections|version=Commercial\u0020with\u0020expiration|versionNumber=2022x|licenseType=Floating|keyVersion=30|protected=fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff|userLimit=10|user=GENERAL\u0020DYNAMICS\u0020MISSION\u0020SYSTEMS\u0020INC|productType=Application|
  floating license

    license-server002$ license-server002 JavaDisplay (v2022) (license-server01/4100 4501), start Sun 9/24 13:21
    root ip-.compute.internal JavaDisplay (v2022) (license-server01/4100 15509), start Wed 9/27 13:16

++++++++++++++
[~ flexlm_exporter]# ./lmutil lmstat -c 4100@l-server01 -i
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Mon 10/2/2023 20:46

NOTE: lmstat -i does not give information from the server,
      but only reads the license file.  For this reason,
      lmstat -a is recommended instead.

Feature                         Version     #licenses    Vendor        Expires
_______                         _________   _________    ______        ________
CC888888Connections             2022         2           cameo         02-jun-2025
CC999999Connections              2022         1           cameo         02-jun-2025
CC000000Connections              2022         1           cameo         02-jun-2025

=======================
[~ flexlm_exporter]# ./lmutil lmstat -c 1055@appserver99 -i
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
Mon 10/2/2023 20:55

NOTE: lmstat -i does not give information from the server,
      but only reads the license file.  For this reason,
      lmstat -a is recommended instead.

 All Rights Reserved.
Mon 10/2/2023 21:34

NOTE: lmstat -i does not give information from the server,
      but only reads the license file.  For this reason,
      lmstat -a is recommended instead.

 All Rights Reserved.
Flexible License Manager status on Mon 10/2/2023 21:34

License server status: 19353@license-server01
    License file(s) on license-server01: /data/app/server/config/server_license.lic:

license-server01: license server UP (MASTER) v11.16.2

Vendor daemon status (on license-server01):

   ibmratl: UP v11.16.2
Feature usage info:

Users of TLSTOK:  (Total of 2400 licenses issued;  Total of 1050 licenses in use)

  "TLSTOK" v2.0, vendor: ibmratl, expiry: 30-dec-2026
  floating license

    svc.useris.JazzServic eadc-e-clmprd03 /dev/tty #u#svc.useris.JazzServic#u##s#JAZZ(77847c02-c045-4a34-9ea2-c6c2d695d55a)#s##p#Rational Quality Manager#p##jazzrole#QualityProfessional#jazzrole# (v2.0) (license-server01/19353 101), start Sun 9/17 11:31, 10 licenses  (linger: 0 / 1800)
    user.last1 eadc-e-clmprd03 /dev/tty #u#user.last1#u##s#JAZZ(fa050840-e4d7-42b7-b4bf-3505277870cd)#s##p#Rational Team Concert#p##jazzrole#Contributor#jazzrole# (v2.0) (license-server01/19353 201), start Sun 9/17 11:31, 5 licenses  (linger: 0 / 1800)
    another-user2 mmmmmm-appp02 mmmmmm-appp02 ClearCase (v2.0) (license-server01/19353 701), start Sun 9/17 11:31, 5 licenses  (linger: 0 / 1800)
    LQE Functional User Key another-server01 another-server01 DOORS#u#LQE Functional User Key#u# (v1.0) (license-server01/19353 13402), start Mon 10/2 5:08, 10 licenses

Running in a container gives me the same issue. Same flexlm flags as above
Updating the lmutil to:
lmutil lmstat -v
lmutil - Copyright (c) 1989-2021 Flexera. All Rights Reserved.
lmstat v<IP_ADDRESS> build 279526 x64_lsb

Gives the same issue, no meaningful logs either:
ts=2023-10-04T18:17:47.691Z caller=flexlm_exporter.go:199 level=info msg="Starting flexlm_exporter" version="(version=, branch=non-git, revision=non-git)"
ts=2023-10-04T18:17:47.691Z caller=flexlm_exporter.go:200 level=info msg="Build context" build_context="(go=go1.20.7, platform=linux/amd64, user=app@918d31421ff1, date=20231002-16:54:34, tags=netgo static_build)"
ts=2023-10-04T18:17:47.691Z caller=flexlm_exporter.go:123 level=info msg="Enabled collectors"
ts=2023-10-04T18:17:47.692Z caller=config.go:48 level=info msg="Loading license config file:"
ts=2023-10-04T18:17:47.692Z caller=config.go:49 level=info -=/home/app/config/licenses.yml
ts=2023-10-04T18:17:47.692Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9319
ts=2023-10-04T18:17:47.692Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9319

It's almost like the the exporter isn't even trying to execute the lmutil binary.  Running the lmutil lmstat commands run fine by itself

@cloudrck about the last case, the container will try to execute lmutil only when you scrape the port 9319, and not before.
Dont you see any error in the logs after some scrapes?

@mjtrangoni Accessing the /metrics endpoint on that port just shows the generic GO GC metrics. No error messages from the exporter

I do have the same problem. Any update on this?

I found the problem but as I'm not a go developer, I can't solve it.
The collectors are not "loaded". The flag variable In line 78 of collector.go is false.
Then in  line 124 of collector.go the if !*enabled results to true and the collecor isn't loaded.
Commenting lines 124-126 in collector.go solves the problem. When these lines are commented, the collectors are loaded and everything works as expected.
So the line 78 in collector.go has to be analyzed why it returns false.
Maybe the bug comes from an update of kingpin?
I've also noticed that the flags collector.lmstat and collectors.lmstat_feature_exp are not available as kingpin flags.
The line 78 in collector.go defines flags for each collector to be enabled/disabled from command line.
Calling  ./flexlm_exporter --collector.lmstat='false'  gives an error. In my understanding there should be flags to enable/disable the collectors from command line.
Hope this helps for fixing this bug.
Cheers, Maurus

@TheRaspyDev  @cloudrck Please confirm if the issue is fixes for everyone, and I will create a new release/ new images as well.

@cloudrck With this patch, I cant reproduce the segmentation fault you are pointing to. How are you building and testing it?

For building: I'm using the docker image golang:1.20.7-bookworm, and having the entrypoint run make to build the flexlm binary.
It's being ran in k8s.
I'll try and build it different ways and see what happens

@TheRaspyDev @cloudrck Please confirm if the issue is fixes for everyone, and I will create a new release/ new images as well.

@mjtrangoni: Works with the new docker image. Thanks a lot!

@mjtrangoni I found the nil pointer reference error only happens with one specific license server. When I remove it, the exporter works fine. And when it is the only server in the config it fails.
The error:
github.com/mjtrangoni/flexlm_exporter/collector.parseLmstatLicenseInfoServer({0xc000966000, 0x25a, 0x8000?})
        /home/myuser/app/collector/lmstat.go:233 +0x232
github.com/mjtrangoni/flexlm_exporter/collector.(*lmstatCollector).collect(0xc000206730, 0xc00060bf78, 0xc0001cecc0?)
        /home/myuser/app/collector/lmstat.go:426 +0x25c
github.com/mjtrangoni/flexlm_exporter/collector.(*lmstatCollector).getLmstatLicensesInfo.func1({{0xc000226834, 0x5}, {0x0, 0x0}, {0xc000238258, 0x16}, {0x0, 0x0}, {0x0, 0x0}, ...})
        /home/myuser/app/collector/lmstat.go:388 +0x85
created by github.com/mjtrangoni/flexlm_exporter/collector.(*lmstatCollector).getLmstatLicensesInfo
        /home/myuser/app/collector/lmstat.go:385 +0xa9

On that line  it appears to be referencing this regex
This particular license sever is the only one that returns multiple license server UP
~$ ./lmutil lmstat -c 1055@problem-server -a
lmutil - Copyright (c) 1989-2018 Flexera. All Rights Reserved.
 All Rights Reserved.
Thu 11/9/2023 21:06

NOTE: lmstat -i does not give information from the server,
      but only reads the license file.  For this reason,
      lmstat -a is recommended instead.

Feature                         Version     #licenses    Vendor        Expires
_______                         _________   _________    ______        ________
a_spaceclaim_dirmod             9999.9999    2           ansyslmd      31-may-2024
a_spaceclaim_dirmod             9999.9999    3           ansyslmd      31-may-2024
a_spaceclaim_dirmod             9999.9999    3           ansyslmd      31-may-2024

@cloudrck nice, this is pointing in the right direction!
But, I fear the problem is related to the hostname itself and not to the rest of the file, or some other special character over there.
Would you mind sending me lmutil lmstat -c 1055@problem-server -a again? Anonymized but in a more transparent way, so that you replace the same type of symbol with something else?
Another option would be sending the original output to me privately at<EMAIL_ADDRESS>Or if you want to give it a try on your side, there is a windows test case, collector/fixtures/lmstat_server_up_win.txt.  Replacing the file with your output should point in the right direction running make test.
Thank you!


Sticky columns

Hi,
Could you please add a feature for sticky columns?
We use a column for buttons and want it to be on the right, but when we show a lot of columns it gets pushed aside.
You can check out fixed columns example in this page for better understanding:
https://ant.design/components/table/

@eylonronen we are working on fixed header: https://github.com/react-bootstrap-table/react-bootstrap-table2/pull/645
However, we dont have plan for sticky column. PR is welcome, thanks

Need sticky column, is there any workaround or this feature implemented ?

this PR may solve the issue. I submitted it to solve the issue for us:
https://github.com/react-bootstrap-table/react-bootstrap-table2/issues/1432


Create es.js

Traduction to Spanish (Latin America)
en.js file convention continued

Hi, @juan-pinzon
Thank you for your contribution!!


Document pip configuration section in project.yml file

Description
Added documentation regarding the pip configuration section in the project.yml file.
Part of #5976
Self Check:

[x] Attached issue to pull request
[x] Changelog entry
[x] Type annotations are present
[x] Code is clear and sufficiently documented
[x] No (preventable) type errors (check using make mypy or make mypy-diff)
[x] Sufficient test cases (reproduces the bug/tests the requested feature)
[x] Correct, in line with design
[x] End user documentation is included or an issue is created for end-user documentation (add ref to issue here: )
[ ] If this PR fixes a race condition in the test suite, also push the fix to the relevant stable branche(s) (see test-fixes for more info)

Processing this pull request

Merged into branches master in 09930d94f66ab663282158b8ea1d74450af96c5d

Processing #6154.


Razor SDK project build-up-to-date check is wrong

VS has its own "up to date" logic that doesn't take Razor scenarios into account when using RazorCompileOnBuild=true which will become the default in 2.1.0-preview2.
The effect of this is that a change in a .cshtml file won't mark the project as 'out of date' as hoped. You have to trigger a rebuild of all dependent projects for things to function correctly.
We need to do some project system work to make this happen.

Apparently we don't need to do anything complicated here. We can add the razor files to UpToDateCheckInput - which will plug them in to the built-in up-to-date check.


ISPN-5698 & 	ISPN-5684

https://issues.jboss.org/browse/ISPN-5698
https://issues.jboss.org/browse/ISPN-5684

@danberindei Fixed all, also updated 7.2.x branch accordingly.

Pulling

@galderz I'm getting a bunch of OOMEs here
Failed tests:
PingOnStartupTest.testGetCacheWithPingOnStartupDisabledMultipleNodes:101 » OutOfMemory
PingOnStartupTest.testGetCacheWorksIfNodeDown » OutOfMemory Java heap space
PingOnStartupTest.testGetCacheWorksIfNodeNotDown » OutOfMemory Java heap space
PingOnStartupTest.testTopologyFetched » OutOfMemory Java heap space
DistFailOverRemoteIteratorTest>BaseIterationFailOverTest.testFailOver:38 » OutOfMemory
org.infinispan.client.hotrod.impl.iteration.MultiServerDistRemoteIteratorTest.clear(org.infinispan.client.hotrod.impl.iteration.MultiServerDistRemoteIteratorTest)
Run 1: MultiServerDistRemoteIteratorTest>BaseMultiServerRemoteIteratorTest.clear:37->BaseMultiServerRemoteIteratorTest.lambda$clear$1:37 » OutOfMemory
Run 2: PASS
Run 3: PASS
Run 4: PASS
Run 5: PASS
org.infinispan.client.hotrod.marshall.EmbeddedCompatTest.createBeforeMethod(org.infinispan.client.hotrod.marshall.EmbeddedCompatTest)
Run 1: EmbeddedCompatTest>SingleCacheManagerTest.createBeforeMethod:55->SingleCacheManagerTest.setup:31->createCacheManager:86 » OutOfMemory
Run 2: PASS
Run 3: PASS
Run 4: PASS
Run 5: PASS

@tristantarrant That's very strange...

@tristantarrant CI looks fine

The OOME has been happening to me as well since a few of weeks, but not on every machine I run the build

Pushed


MacOS: NoSuchMethodError with 1.0.3

The error didn't happen with previous version, but the latest version 1.0.3 encountered problem when runs on MacOS, tested with both jdk 17 and 22,  attached the core dump of the crash:
hs_err_pid3748.log
Perhaps problem with the embedded native lib for mac?

Fixed, wait for a few hours, after maven central updates, you will beable to use latest version v1.0.4.
That's the latest version test logs on macos platform:
u@udeMac build % java -jar StableDiffusionConsole-1.0.3.jar
Using CPU backend
loading model from '/Users/u/Desktop/models/sd-v1-4.ckpt'
load /Users/u/Desktop/models/sd-v1-4.ckpt using checkpoint format
init from '/Users/u/Desktop/models/sd-v1-4.ckpt'
Stable Diffusion 1.x
Stable Diffusion weight type: f32
loading vocab
vocab size: 49408
ggml tensor size = 416 bytes
clip params backend buffer size =  470.66 MB (449 tensors)
unet params backend buffer size =  2165.24 MB (720 tensors)
vae params backend buffer size =  179.03 MB (328 tensors)
preparing memory for the weights
loading weights
loading tensors from /Users/u/Desktop/models/sd-v1-4.ckpt
model size = 2784.45MB
total memory buffer size = 2814.93MB (clip 470.66MB, unet 2165.24MB, vae 179.03MB)
loading model from '/Users/u/Desktop/models/sd-v1-4.ckpt' completed, taking 14.99s
running in eps-prediction mode
finished loaded file
load:true
prompt after extract and remove lora: "test"
apply_loras completed, taking 0.00s
parse 'test' to [['test', 1], ]
split prompt "test" to tokens ["test", ]
learned condition compute buffer size: 2.58 MB
computing condition graph completed, taking 96 ms
parse '' to [['', 1], ]
split prompt "" to tokens []
learned condition compute buffer size: 2.58 MB
computing condition graph completed, taking 89 ms
get_learned_condition completed, taking 187 ms
sampling using Euler A method
generating image: 1/1 - seed 42
diffusion compute buffer size: 552.28 MB
|>                                                 | 0/20 - 0.00it/s
|==>                                               | 1/20 - 21.48s/it

@javaaiorg  Works perfectly, thanks for fixing it!

@javaaiorg Weird, I am using 1.0.4 and seems like the problem is back again, I see NoSuchMethodError again in the dump file after the program crashed.

I have tested it on jdk8 and jdk11, and the image can be generated successfully. Can you give me your call stack? And tell me the jdk version you are using?

@javaaiorg Sure, I am using jdk17 and here's the log file:
hs_err_pid18921.log

@javaaiorg If it helps, I am running on Sonoma 14.5.


Make append(DSL) ignore value which inserted before

Summary
I propose append dsl makes array to be unique.
This proposal makes below situations to be avoidable:
# in capistrano-puma gem
append :rbenv_map_bins, 'pumactl'

# deploy.rb
append :rbenv_map_bins, 'pumactl'

$HOME/.rbenv/bin/rbenv exec $HOME/.rbenv/bin/rbenv exec bundle exec bundle exec pumactl -S /home/deploy/application/shared/tmp/pids/puma.state -F ...

Please notice that command has duplicated rbenv exec.
I think these kind of problem don't occur frequently but hard to find. and also, until append introduced, there is gems used set accidently (I believe).
This problem may occur when a user tries to update gems which contains patch replacing set to append.
One more, this makes plugin developer do not care about the other plugins, especially duplicated adding configuration variable.
Short checklist

[x] Did you run bundle exec rubocop -a to fix linter issues?
[x] If relevant, did you create a test?
[x] Did you confirm that the RSpec tests pass?
[ ] If you are fixing a bug or introducing a new feature, did you add a CHANGELOG entry?

I will do when this proposal confirmed. XD

Thanks to read this. Have a nice day :)

Interesting idea. I can't think of a situation where I would want the array to include duplicates, but from my experience maintaining this project, the instant we make that not possible, someone else will come tell us about their workflow, which we just broke…

I wonder whether we simply need to be more specific about how #append works. Right now, it looks like:
set(key, Array(fetch(key)).concat(values))

Which coerces into an array, and then adds a value. We may just want to only operate on arrays (or array-like things, like Set) and fail if it isn't an array. Then we could handle Array and Set and change capistrano/rbenv to use Set instead since it doesn't really make sense to duplicate binary names in this case.
Other maintainers: thoughts?

Another fix is to make capistrano/rbenv more resilient to this.
This https://github.com/capistrano/rbenv/blob/master/lib/capistrano/tasks/rbenv.rake#L22 could be changed to uniq the array before iterating over it.
Or maybe go further back and make SSHKit.config.command_map not allow duplicates.

Hmm. I don't think we should change Capistrano's behavior in this case. The method is called append, and it would be surprising/misleading if behind the scenes it is actually doing something different and more complicated.
I think the responsibility lies in capistrano/rbenv to call uniq.

Alternatively we could define a new DSL method called add that follows set semantics. (Not saying that's a good idea, but putting it out there.)

Thanks to quick response. :)

The method is called append, and it would be surprising/misleading if behind the scenes it is actually doing something different and more complicated. I think the responsibility lies in capistrano/rbenv to call uniq.

I agree with proposed behavior may make misleading, however, for example, not only capistrano/rbenv, could we say to the maintainers of whole plugin which want to use append like 'ok, append don't grant uniqueness, so you check uniqueness when use append variable if you need'? Instead of this, It's better to provide something which grant uniqueness I think, also it helps many plugin works more reliable from another plugin or user's mistake.
I don't say this proposal accepted, but we may need some way to keep uniqueness of enumerable.
@will-in-wi 's ideas seems good to me, but It seems to be better that define new dsl for this. Explicit way makes better understanding how to use capistrano, but also makes it easy where to dig if we have problem related this.


Add a state barrier for getLatestStableLog

What is the context for this pull request?

Tracking Issue: n/a
Parent Issue: n/a
Dependencies: n/a

What changes were proposed in this pull request?

In getLatestStableLog, stop searching the previous logs if there's creating or vacuuming state.
Since logs before creating or vacuuming state are irrelevant to the current index and invalid.
Does this PR introduce any user-facing change?

Yes
How was this patch tested?

Unit test

minor comments otherwise LGTM 👍 thanks @sezruby