REST API response with /v1/unlockwallet

Background
I try to automate the unlocking of the wallet. I do this using the REST API and the /v1/unkockwallet endpoint.
As a result, I expected a HTTP response with a 200 (OK) status code.
I received 503 instead with the body:
{"error":"transport is closing","message":"transport is closing","code":14}

However, the wallet still was successfully unlocked.
Your environment

v0.9.0-beta
I run lnd within a docker container based on golang:1.12.13-alpine3.10.

Linux 17649df81af1 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 Linux

bitcoind version is  v<IP_ADDRESS>.

Steps to reproduce
I start lnd and wait until it wants me to unlock the wallet.
I do:
curl -i -X POST --cacert /data/lnd/tls.cert   https://some_domain:8080/v1/unlockwallet -d '{"wallet_password": base64_encoded_passwd}'

Expected behaviour
A HTTP response with a status code 200.
And maybe a body in JSON format that says {"result": "success"}.
Actual behaviour
I receive:
HTTP/1.1 503 Service Unavailable
Content-Type: application/json
Date: Tue, 04 Feb 2020 16:27:25 GMT
Content-Length: 75

{"error":"transport is closing","message":"transport is closing","code":14}

And the wallet gets unlocked.
I even tried including the default readonly.macaroon which gave the same status code 503 but did not return a body:
HTTP/1.1 503 Service Unavailable
Content-Type: application/json
Date: Wed, 05 Feb 2020 13:52:51 GMT
Content-Length: 75

Also, when providing a wrong password, the REST server returns:
HTTP/1.1 500 Internal Server Error
Content-Type: application/json
Trailer: Grpc-Trailer-Content-Type
Date: Wed, 05 Feb 2020 13:51:20 GMT
Transfer-Encoding: chunked

{"error":"invalid passphrase for master public key","message":"invalid passphrase for master public key","code":2}

which is fine in general. I would prefer another status code, though. Something like 400 maybe (just my personal preference).

Does this happen always or just sometimes? And what does the log of lnd say when this occurs? Could be a timing issue, because the WalletUnlocker is shut down just after the wallet is unlocked. I think this is related to #3982 and can probably be fixed at the same time.

I was able to reproduce the issue locally. It happens every time.
Here is my lnd log, before and after getting the {"error":"transport is closing","message":"transport is closing","code":14} response from the REST API:
2020-02-05 17:26:26.848 [INF] LTND: Version: 0.9.0-beta commit=v0.9.0-beta, build=production, logging=default
2020-02-05 17:26:26.848 [INF] LTND: Active chain: Bitcoin (network=testnet)
2020-02-05 17:26:27.492 [INF] CHDB: Checking for schema update: latest_version=12, db_version=12
2020-02-05 17:26:27.504 [INF] RPCS: password RPC server listening on <IP_ADDRESS>:10009
2020-02-05 17:26:27.505 [INF] RPCS: password gRPC proxy started at <IP_ADDRESS>:8080
2020-02-05 17:26:27.505 [INF] LTND: Waiting for wallet encryption password. Use `lncli create` to create a wallet, `lncli unlock` to unlock an existing wallet, or `lncli changepassword` to change the password of an existing wallet and unlock it.
2020-02-05 17:26:38.614 [INF] LNWL: Opened wallet
2020-02-05 17:26:38.615 [INF] LTND: Primary chain is set to: bitcoin
2020-02-05 17:26:38.636 [INF] LTND: Initializing bitcoind backed fee estimator
2020-02-05 17:26:38.636 [INF] LNWL: Started listening for bitcoind block notifications via ZMQ on <IP_ADDRESS>:28332
2020-02-05 17:26:38.636 [INF] LNWL: Started listening for bitcoind transaction notifications via ZMQ on <IP_ADDRESS>:28333

In case it is helpful, I tried the same with an older version of lnd (0.8.1) the behavior is the same but the response I get is different:
curl -i -X POST --cacert ~/.lnd/tls.cert https://localhost:8080/v1/unlockwallet -d '{"wallet_password": "<wallet-password-base64>"}'
HTTP/1.1 408 Request Timeout
Content-Type: application/json
Date: Wed, 05 Feb 2020 16:59:30 GMT
Content-Length: 37

{"error":"context canceled","code":1}

Could be a timing issue, because the WalletUnlocker is shut down just after the wallet is unlocked. I think this is related to #3982 and can probably be fixed at the same tim

Yeh I think this is the issue. Perhaps we can try to block until the message is received by the other end then shutdown the unlocker service? Long-term I think we aim to keep the service running at all times so users can easily tell if the wallet is unlocked or not.

Thanks @dethos and @guggero
Let's wait for the https://github.com/lightningnetwork/lnd/issues/3982 and test again.

This should be fixed by #3982. Feel free to reopen if this still happens.


Fixed bug in 'Custom Servers' section

Coverage remained the same at 89.189% when pulling 1978bdafdf35384f676585787623eff4a410e970 on robfors:master into 2fd757b5dfbeb4c55b0985a2c322793f54884bab on socketry:master.

What was the bug?

The first issue I found was MyServer.run needs to be MyServer.new.run.
Then I tried to run
class MyServer < Async::DNS::Server
	def process(name, resource_class, transaction)
		transaction.fail!(:NXDomain)
	end
end

Async::Reactor.run do
	task = MyServer.run
	task.stop
end

with little success. I tried to sleep and such but it would not work. Then found that I could just remove task.stop it worked. So I backtraced my commit

I see. Thanks for your PR.
Let me clear some things up for you.
Imagine that the flow control inside the reactor block is linear and non-blocking. If you start the server and then call stop, it will stop immediately.
That's the expected behaviour.
If you want to wait on the server to finish, you can do the following:

If you want to be explicit, you can call task.wait. But as it's a server it would never stop in this case, unless it crash or some other problem.

If you actually want to sleep indefinitely, call Async::Reactor.run do |task|; ... task.sleep.

Async works like normal flow control as much as possible. But there are some limitations due to MRI/Ruby. (e.g. sleep vs task.sleep).
For the vast majority of servers, all you need to do is use Async::Reactor.run do; MyServer.new.run; end the example simply shows how you can shut down the server by calling stop, but perhaps that line should be removed/commented out because it's not typical flow control in a server (except perhaps in tests/specs).

After thinking about what you said, I updated README a bit more to clarify when to use task.stop. Thanks so much for taking the time to help the project :)

Thanks for the help. Async seems pretty cool once you understand it.

Thanks I appreciate your enthusiasm. The first commit to RubyDNS was in 2009, so it's been an interesting journey, async/async-dns and a few other bits were the end result.


Error: Token expected but "JSXAttribute" found

Hello! I'm getting the error: Error: Token expected but "JSXAttribute" found when parsing the following JSX: <input type="text" id="input" />. When I remove id attribute, parsing is done without errors.
Is it a known bug or am I doing wrong something?
Here's a snippet of code I'm running.
var Parser = require('cst').Parser;
var parser = new Parser();

parser.parse('<input type="text" id="input" />');

Hello!
This is probably a bug. I will look at this soon, thank you.

Fixed in 0.0.17

Wow my bad that seems like an obvious test

Thanks! 👍


Fatal Exception: java.lang.IllegalStateException: Reply already submitted (Is not fixed)

According to PR #360 this bug was fixed, but in fact it's still happening. I have updated to the last version yesterday, and released to production. Today I got this error from Firebase Crashlytics several times, which I was not getting before on version (^5.1.2) ( I got it a few times, but this time is worse).
Error from Firebase Crashlytics:
Fatal Exception: java.lang.IllegalStateException: Reply already submitted
       at io.flutter.embedding.engine.dart.DartMessenger$Reply.reply(DartMessenger.java:430)
       at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler$1.success(MethodChannel.java:267)
       at com.dooboolab.flutterinapppurchase.MethodResultWrapper$1.run(MethodResultWrapper.java:27)
       at android.os.Handler.handleCallback(Handler.java:938)
       at android.os.Handler.dispatchMessage(Handler.java:99)
       at android.os.Looper.loop(Looper.java:223)
       at android.app.ActivityThread.main(ActivityThread.java:7955)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:603)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)

Crash on: Android (real device)
flutter_inapp_purchase: ^5.2.0
flutter stable channel: ">=2.12.0 <3.0.0"
Actual Behaviour:
My app has a (weekly/monthly) subscription feature, which I deliver to the user if the payment is successful. Every time the user access a premium content, I get his purchase history, and I check if he has an active subscription, if he does not have, I suspend the premium feature in the app. But when I use the .getAvailablePurchases() method, I get an error and the app crashes.
@valterh4ck3r @offline-first Tried to fix this error on the PR #360, but it's still happening. Maybe @hyochan should downgrade this version.

I have this error on 5.1.2

@meowofficial , please update your package to 5.2.0 and test again.

@meowofficial Do you still have the same bug on version 5.2.0? Because I do.

@valterh4ck3r @soarescaique Yes I tested 5.2.0 on last beta and stable channels, but app crashes because of getProducts() and getSubscriptions() (getAvailablePurchases() works as expected). When I call this methods with open connection app crashes. When I call this methods without open connection I get PlatformException(getItemsByType, IAP not prepared. Check if Google Play service is available., , null)

@soarescaique I didn't test example code. getAvailablePurchases() doesn't crash the app for me at least when it returns empty list. It crash the app when I try to fetch products or subscriptions on Android.

@meowofficial I would suggest you to try the example code and see if it works as expected. Cause I use this package for a few months on production and I never had this error when trying to fetch products ou subscriptions. But anything is possible. Check the example code and let me know.

@soarescaique can you test with this example on real devices?
void test()async{
    await FlutterInappPurchase.instance.initialize();
    _testItems();
    _testPurchases();
    _testHistory();
  }

  void _testItems()async{
    final subscriptions = FlutterInappPurchase.instance.getSubscriptions(["1_month", "12_month"]);
    final products = FlutterInappPurchase.instance.getProducts(["plus_lifetime"]);
    final itemsFutures = await Future.wait([subscriptions, products]);
    final items = itemsFutures.expand((it) => it).toList();
    items.forEach((it) { debugPrint("TEST items:${it.toDebugString()}");});
  }

  void _testPurchases()async{
    final purchases = await FlutterInappPurchase.instance.getAvailablePurchases();
    if(purchases == null || purchases.isEmpty)debugPrint("no purchases");
    purchases?.forEach((it) { debugPrint("TEST getAvailablePurchases:${it.toDebugString()}");});
  }

  void _testHistory()async{
    final history = await FlutterInappPurchase.instance.getPurchaseHistory();
    if(history == null || history.isEmpty)debugPrint("TEST no history");
    history?.forEach((it) { debugPrint("TEST getPurchaseHistory:${it.toDebugString()}");});
  }

If you have any issues, please test with this PR #365

@offline-first I will do it and let you know the tests result. Hope it works.

@offline-first Sorry for the late response. I just tried your example above and got the same error.
I am running on version 5.2.0.
Tested on a Real Android Device.
Got the error after calling .getAvailablePurchases()
Fatal Exception: java.lang.IllegalStateException: Reply already submitted
       at io.flutter.embedding.engine.dart.DartMessenger$Reply.reply(DartMessenger.java:430)
       at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler$1.success(MethodChannel.java:267)
       at com.dooboolab.flutterinapppurchase.MethodResultWrapper$1.run(MethodResultWrapper.java:27)
       at android.os.Handler.handleCallback(Handler.java:938)
       at android.os.Handler.dispatchMessage(Handler.java:99)
       at android.os.Looper.loop(Looper.java:223)
       at android.app.ActivityThread.main(ActivityThread.java:7955)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:603)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)

Thank you!
I will fix other issues with the android plugin in the next days and add more features.
Hopefully the author of this plugin check #365

@offline-first Thank you for your time and contribution. Feel free to contact me to help testing your changes or anything. I was making some changes on the .java files of the package to make it work, and then submit a PR. But since you have refactored the .java files to .kotlin I will be waiting for @hyochan to approve it, and then add some features too.


Possibility to move between years on the picker

Hi,
when we have a birth date field, is slow to navigate back in time a month at a time.
Is there a way or is going to be a way to move between years?
Best regards
Rui Santos

You can move between years with the option selectYears: true  on initialization. You can specify a large range of years instead of true to make jumping back easier.
Someone else would know hopefully if there is a way to preload the calendar for past years. I imagine for a birthday it doesn't make sense to open the calendar initially to 2015 right?


fix(interaction): 修复默认隐藏列的配置更新为空数组时, 未触发表格更新

👀 PR includes

🐛 Bugfix

[x] Solve the issue and close #0

📝 Description
比如: 先隐藏 A, 再展开 A, hiddenColumnFields 为空时也应该重新渲染, 清空上一次渲染的展开icon 和 详情信息
sheet.setOptions({
  interaction: {
    hiddenColumnFields: ['A']
  }
})

sheet.setOptions({
  interaction: {
    hiddenColumnFields: []
  }
})

其他:

去除 React 层多余的一次更新
去掉 Switcher 组件的 JSON.stringify 比较

🖼 Screenshot

Before
After

🔗 Related issue link

🔍 Self Check before Merge

[x] Add or update relevant Docs.
[x] Add or update relevant Demos.
[x] Add or update relevant TypeScript definitions.

:tada: This PR is included in version @antv/s2-v1.18.0-alpha.3 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @antv/s2-react-v1.16.0-alpha.3 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @antv/s2-v1.18.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @antv/s2-v1.18.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @antv/s2-react-v1.16.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Introduce default var confluence_download_directory

Changes:
Introduce default var confluence_download_directory to support overriding the download path at role usage level.

Refer to https://github.com/alvistack/ansible-role-bitbucket/pull/1, let's close this now.


what should I do if I want to apply some changes to plugins like action?

I'v add _system_impl->set_flight_mode(FlightMode::Posctl);  to arm() function to test if i can apply changes to plugin implications,while hoping that when arm() is called MAVSDK can send messages to switch the UAV's flight mode to position.
So,what should i do to build the MAVSDK to apply my changes?I followed the guidance in contribution to and ran auto-generation and building library from source but it did work,have i missed anything?I am grateful for any guidance and help.

This goes back to the feature request for a "set mode" API, I'd say.
Also see: #2128

(You can always do any changes and build yourself but I usually don't recommend that as you are then on a fork that you need to maintain yourself.)

thanks,i'll be careful,there will not be a large difference,just some adaption to my self-built autopilot

(You can always do any changes and build yourself but I usually don't recommend that as you are then on a fork that you need to maintain yourself.)

I'v solved to problem thanks a lot.


[Component.Resources] :android:assembleDebug and :desktop:run raises error in a multi-module environment

Describe the bug
In a multi-module environment, Android and Desktop apps will fail to run if multiple strings.xml are placed for each module.
Stacktrace(Android)
> Task :android:mergeDebugJavaResource FAILED

Execution failed for task ':android:mergeDebugJavaResource'.
> A failure occurred while executing com.android.build.gradle.internal.tasks.MergeJavaResWorkAction
   > 2 files found with path 'composeRes/values/strings.xml'.
     Adding a packagingOptions block may help, please refer to
     https://developer.android.com/reference/tools/gradle-api/8.2/com/android/build/api/dsl/ResourcesPackagingOptions
     for more information

...

Stacktrace(Desktop)
> Task :desktop:run
Exception in thread "main" java.lang.IllegalStateException: String ID=`module2_text` is not found!
	at org.jetbrains.compose.resources.StringResourcesKt.loadString(StringResources.kt:102)
	at org.jetbrains.compose.resources.StringResourcesKt.access$loadString(StringResources.kt:1)
	at org.jetbrains.compose.resources.StringResourcesKt$stringResource$str$3.invokeSuspend(StringResources.kt:77)
	at org.jetbrains.compose.resources.StringResourcesKt$stringResource$str$3.invoke(StringResources.kt)
	at org.jetbrains.compose.resources.StringResourcesKt$stringResource$str$3.invoke(StringResources.kt)
	at
...

Affected platforms
Select one of the platforms below:

Android
Desktop

Versions

Kotlin version: 1.9.21
Compose Multiplatform version: 1.6.0-dev1350

To Reproduce
Clone this repository and run the command :android:assembleDebug or :desktop:run.
Expected behavior
Android and Desktop apps must run correctly.

Yes. At the moment the compose resources doesn't support multimodule projects. It will be fixed in the future.

https://github.com/JetBrains/compose-multiplatform/pull/4454

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Port for..in, for..of and other loops to AskScript

In order to keep minimal resources in AskVM, it would be good to port for..in, for..of, maybe for (and possibly other kinds of loops, if there are any at the time of reading this issue) from AskVM to AskScript and leave in AskVM only while (or for).

Please see the status of this discussion first: https://github.com/xFAANG/askql/issues/302

When porting these to AskScript, one should compare performance of Typescript forIn/forOf resources with their AskScript equivalents.


BUG: Remove nullability from attribute properties

As mentioned in #175, attributes can't have nullable properties. It doesn't compile. It seems you haven't tested it.
That PR used nullable backing fields and GetOrDefault to retrieve them. I think that may actually be better for the future, enabling properties to be set in different levels, but here I took the easy route.
Also note that the default RequestTimeoutMs in this library is 5000ms, whereas in the Confluent library, if not set, it's 30000ms. Since it was impossible to specify RequestTimeoutMs before this fix, this would actually mean that we're changing from 30000 to 5000 now. The documentation for Confluent's MessageSendMaxRetries claims that the default is int.MaxValue, but I doubt that's the truth. https://github.com/confluentinc/confluent-kafka-dotnet/issues/1858
Edit: It was the truth

@shrohilla I would really appreciate a bit of feedback on a bunch of PR:s. This one is a very clear bug that was reported more than 2 years ago.

@gliljas could you please explain more details about this PR, is default value is not being reflected for all attributes or is this only for int max value is assigned to MaxRetries attribute on startup ??

The PR is about having nullable properties on attributes, i.e MaxMessageBytes, BatchSize, EnableIdempotence, MessageTimeoutMs, RequestTimeoutMs amd MaxRetries. It's not allowed. Your code will not compile if you use any of these attributes.

@gliljas could you please share the sample code where you found there was code compilation issue with functions ??
As per my understanding, the default value should be assigned to these properties if they are not set

Well, basically you could just try to use any of the mentioned properties yourself, and you'll see that it doesn't work.
Attribute property values must be compilable to constants. Nullables aren't.
https://dotnetfiddle.net/w5NrwO

@gliljas have you tried this solution if this working fine with the function ??
I doubt assigning the default values only will work

/azp run

@gliljas Thanks alot for your contribution.
There is also regression with the change, I would request you to please fix that.
I'll also request you to please add 1 or 2 E2E tests with the change for both E2E and Lang E2E for testing the regression.

I doubt assigning the default values only will work.

That is not what the code does. The property types have changed. The end result is that the Confluent library will always be populated with the default values specified here, instead of Confluent's default values, so they have to be specified. In the case of MaxRetries, there is a difference, since Confluent specified int.MaxValue but this library specifies 2.
Another option, which I used in a PR two years ago is to use nullable backingfields, which will only get a value if they're specifically set.

I'll check the regression first.

/azp run

@gliljas I would request you to add 1 End to End test case to validate the flow

/azp run

Sure thing, but any suggestion what such a test would do and assert? :)

Sure thing, but any suggestion what such a test would do and assert? :)

One way could be try to re ingest the same event using the output binding after enabling idempotency than producer will not allow to re-ingest the same event and throw the error, that error you need to read from the log -- https://github.com/Azure/azure-functions-kafka-extension/blob/dev/test/Microsoft.Azure.WebJobs.Extensions.Kafka.EndToEndTests/TestLoggerProvider.cs#L42
Please share the feedback if this proposal doesn't works out.

@gliljas any update on this ??

@gliljas any update on this ??

Just making sure it was noted that an e2e test was added. Testing idempotency is a bit too hidden in Kafka to be triggered from the outside, so I chose MaxMessageBytes instead.

/azp run

@gliljas Thanks alot for your contribution we will plan this change in next release.


[Bug Report]: Parses all JS files in strict mode

System Info
  System:
    OS: Linux 6.2 NixOS 23.05 (Stoat) 23.05 (Stoat)
    CPU: (12) x64 Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz
    Memory: 8.50 GB / 15.27 GB
    Container: Yes
    Shell: 5.2.15 - /run/current-system/sw/bin/bash
  Binaries:
    Node: 18.14.2 - ~/.nix-profile/bin/node
    Yarn: 1.22.19 - ~/.nix-profile/bin/yarn
    npm: 9.5.0 - ~/.nix-profile/bin/npm
  Browsers:
    Chromium: 111.0.5563.64
    Firefox: 111.0b8
  npmPackages:
    @rspack/cli: ^0.1.1 => 0.1.1

Details
rspack has inherited SWC’s default of parsing all JS files as modules, and therefore cannot parse code that only compiles in non-strict mode, such as
delete a;

or as a real-world example, the sorttable library.
SWC provides the isModule configuration option for this; I think we want to set it based on the file extension (.cjs, .mjs, .cts, .mts) or the type field in the nearest package.json (.js, .ts).
See for example

swc-project/swc#1810

Probably related:

#2178

Reproduce link
No response
Reproduce Steps
From an empty directory:
$ mkdir src
$ echo 'delete a;' > src/index.js
$ rspack
Hash: eb46cf29de5a8584
PublicPath: auto
  Asset       Size  Chunks             Chunk Names
main.js  248 bytes    main  [emitted]  main
Entrypoint main = main.js
[887] ./src/index.js 10 bytes {main}

error[javascript]: JavaScript parsing error
  ┌─ src/index.js:1:8
  │
1 │ delete a;
  │        ^ 'delete' cannot be called on an identifier in strict mode

build: 29.688ms

This can be solved when SWC parses non-module correctly, you can check the progress on #2406.


Is the settings_sent parameter of the schema strategy used?

In the schema strategy there is an option named settings_sent that does not appear to be used in the code. Is it still somehow used?

I don't think so :)


request for decoration block - breathing dimensional transceiver

Not sure if this is the right place to request a new block.  But here it goes.
Would you be inclined to add a block (for decoration) that looks like the dimensional transceiver and breaths like it?  It would really be cool if it could be chiselable via Chisels & Bits.
thanks!

I don't think c&b can work with models that are not made up from bit-sized voxels, but you'd have to its author about that.
There is a third deco block on the roadmap for the rest of the machines. However, that breathing part is not (and cannot be) part of the block model, so it cannot be recreated by anything else but a dimTrans (or a completely new block that basically would be a dimTrans that doesn't work). So don't expect that to come, sorry.
Sent from my iPad

On 06.12.2016, at 03:59, Ramin Ahangarzadeh<EMAIL_ADDRESS>wrote:
Warning: This message has had one or more attachments removed
Warning: (msg-23255-16.html).
Warning: Please read the "j-e-b-Attachment-Warning.txt" attachment(s) for more information.
Not sure if this is the right place to request a new block.  But here it goes.
Would you be inclined to add a block (for decoration) that looks like the dimensional transceiver and breaths like it?  It would really be cool if it could be chiselable via Chisels & Bits.
thanks!
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/SleepyTrousers/EnderIO/issues/3888
This is a message from the MailScanner E-Mail Virus Protection Service
The original e-mail message contained potentially dangerous content,
which has been removed for your safety.
The content is dangerous as it is often used to spread viruses or to gain
personal or confidential information from you, such as passwords or credit
card numbers.
Due to limitations placed on us by the Regulation of Investigatory Powers
Act 2000, we were unable to keep a copy of the original attachment.
The content filters found this:
MailScanner: Found a script in HTML message
--
Postmaster
MailScanner thanks transtec Computers for their support

Thank you for your suggestion. It has been added to our list or possible enhancement.
This ticket will be closed because we want the list of open tickets to reflect known issues.

Understood.  Thanks for considering it.
Regards,


Add apt_update and apt_repository

I debated making a second cookbook but figure that actually this is probably the right thing to do. cc @lamont-granquist @tas50

👍 that this is the right thing to do, i don't think we need another cookbook


Fixing scipy=1.0.0 incompatibility of lyapunov discovered in PR #573

As discussed in PR #573 scipy bug, scipy/scipy#8155, caused failing import for users of scipy=1.0.0 this is a simply work around to handle this edge-case.
P.s. we should delete kern/_src unless it is used for something I am unaware of.

Codecov Report

Merging #574 into devel will decrease coverage by 2.96%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##            devel     #574      +/-   ##
==========================================
- Coverage   53.04%   50.08%   -2.97%
==========================================
  Files         202      202
  Lines       20195    20197       +2
  Branches     2701     2701
==========================================
- Hits        10713    10115     -598
- Misses       8993     9632     +639
+ Partials      489      450      -39

Strange, am I crazy or is it saying that those 4 lines changed the coverage by ~ -3.2%?
It seems almost all to do with prediction/plotting, could the change to scipy==1.0 be also making a difference here for some reason? @mzwiessele, any idea? Something to do with the plotting tests?
The three lines can't feasibly be tested without modifying travis to test both scipy < 1.0 and scipy == 1.0 which won't be represented in the coverage.


flac tags - don't concatenate artist

FLAC tag fields "Artist" and "Album Artist" support multiple entries (relevant spec).
Don't combine multiple artists into a single string to improve support with many different media playing applications.
fixes #4 and at least one issue in https://github.com/yaronzz/Tidal-Media-Downloader
also matches behavior of https://github.com/yaronzz/Tidal-Media-Downloader-PRO

What's the behaviour of this for end output? How's it work with Plex? Does Plex pickup the first artist or which?
I've got a similar pull relevant to a similar issue, but it instead uses only the MAIN artist so it's friendly for things like plex

What's the behaviour of this for end output? How's it work with Plex? Does Plex pickup the first artist or which?
I've got a similar pull relevant to a similar issue, but it instead uses only the MAIN artist so it's friendly for things like plex

Plex handles multiple artists just fine. My only complaint is that multiple artists are delimited with a ';' in plexamp. This is somewhat ugly, but still usable. The plexamp devs have ignored my request to change it to ', '. I might reach out again with another request.

What's the behaviour of this for end output? How's it work with Plex? Does Plex pickup the first artist or which?
I've got a similar pull relevant to a similar issue, but it instead uses only the MAIN artist so it's friendly for things like plex

Plex handles multiple artists just fine. My only complaint is that multiple artists are delimited with a ';' in plexamp. This is somewhat ugly, but still usable. The plexamp devs have ignored my request to change it to ', '. I might reach out again with another request.

As is the multiple artists (as a string) result in plex thinking it's one artist and making a folder named "artist1, artist2, artist3"... etc


Reconsider the value of named container ports

Allowing a service to target a port by name-or-number brings some complexity to a number of places in our codebase.  Is it worthwhile?  I am no longer convinced.  Discuss..

More detail: IntOrString is used in Service.ContainerPort and in HTTPGetAction and TCPSocketAction.
I'm in favor of getting rid of IntOrString for Service.ContainerPort.
It seems less of a problem for the hooks/probes, and would be more tedious to keep those port values in sync.

If we keep named ports at all it feels bad to not support them
consistently..
On Mon, Feb 23, 2015 at 1:23 PM, Brian Grant<EMAIL_ADDRESS>wrote:

More detail: IntOrString is used in Service.ContainerPort and in
HTTPGetAction and TCPSocketAction.
I'm in favor of getting rid of IntOrString for Service.ContainerPort.
It seems less of a problem for the hooks/probes, and would be more tedious
to keep those port values in sync.
--
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/issues/4737#issuecomment-75636747
.

Fair enough. At least all ports in a pod could be updated atomically.
We may want to keep port names for informational purposes if nothing else.

I think we agreed to change the ones in containers to descriptions.

keeping


[WIP] Get publisher's OS Image details

Along with commands for replicating and publishing (#497), and updating extended OSImage attributes (#501), fetching additional publisher-specific details closes the loop for publishers; allowing for clients to monitor replication progress, look up all published images in a particular image family, or list out SKUs.

@lmazuel if you could contribute a recording for the additional test, I would appreciate the help.
If not, I'll have one here... eventually ;-)

I need to do some more work here; I'll open another PR when it's in better shape.


Added the ability for [OBJECT_ID] replacement when the validations are an array

The validation won't work when using this piece of code.
eloquent('userName', 'email')->ensure(['required', 'unique:users,email,[OBJECT_ID]']),

This PR fixes that.
While I'm here, is there a better way to do this? I know a 409 should be returned for duplicates.

Never mind. Using closure validation is probably better.


TabsPanel changes width upon first change of the tab

May not be related to this PR, but just noticed some shift in the width of the tabs panel in the "Vertical Overflow" story
14.webm

Originally posted by @rohan-kadkol in https://github.com/iTwin/iTwinUI/pull/1548#discussion_r1355274285

I am on a Windows 11 + Edge 117. What I did was:

Press tab to focus the selected "Item 11`"
Press up or down

Originally posted by @rohan-kadkol in https://github.com/iTwin/iTwinUI/pull/1548#discussion_r1355294539

I still cannot reproduce on Mac, so would be nice if @gretanausedaite can verify and add repro steps.

I can reproduce it.
It seems this visual bug is present only on first mount.

I tried finding the cause for this width jump but unfortunately still couldn't find it. So I thought to share my observations to get others' ideas.

Detailed screen recordings of the issue
On Chrome/Edge, I noticed that in the Vertical Tabs, whenever a scroll bar is present (when vertical overflow), then there are some jumps in the width.
https://github.com/iTwin/iTwinUI/assets/45748283/64eaafa8-c5c9-41f7-85b0-1e5e0b38da94
0:00-0:13 = In the horizontal tabs, whenever there is a horizontal overflow, the scrollbar creates place between the tabs list and the tab content. This seems correct.
0:13-0:20 = In the vertical tabs, we first see the non-overflow state. So there is no scrollbar. We also see that the padding for the last very long item is correct. Again, everything looks correct so far.
0:20-0:29 = But when we trigger an overflow, there seems to be some issue with the padding.
0:29-0:38 = Doing the first click on any button creates an abrupt jump in the width. But after that first click, the padding issue gets fixed.
0:38-end = We now notice then whenever we toggle the "show all" button, regardless of whether the vertical tabs overflows or not, the initial render always seems to have some padding issue that is always fixed upon the first click

On Firefox, the horizontal tabs work the same way as Chrome.
https://github.com/iTwin/iTwinUI/assets/45748283/dfc1289a-043e-4fad-8817-008a60a0902a
The vertical tabs gives the same padding issue as Edge/Chrome. However, this time, clicking any button for the first time does not fix the padding issue.

Some thoughts/observations:

This issue didn't originate from #1548 (v3). Since this issue can also be seen in the v2 storybook's Tabs -> VerticalOverflow story.
I guess this is a CSS issue since different browsers give different behaviors when trying to repro. Adding scrollbar-gutter: stable to the vertical tabs fixes the problem on all three browsers. However, the gap between the tabs-list and the tabs-content even when there is no scroll is not ideal.
The issue appears even when the use of useContainerWidth() in Tabs is removed.

Some oberservations made jointly with @mayank99:

The jumping in the width upon the first click of any tab appears only some of the times and with some specific data (see repro below for one of those cases).
white-space: nowrap may be contributing to the issue?
The width jump is seen even when the very long tab item is not used.

https://github.com/iTwin/iTwinUI/commit/252c230d07cc6790a58ab8b4edd2e980140bed31: Contains a good repro (in the vite playground) along with some trial and error made jointly with @mayank99.

I was able to stop the layout shift on click, by using this hack on first mount:
const originalOverflow = tablist.style.overflow;
tablist.style.overflow = 'visible';
queueMicrotask(() => {
  tablist.style.overflow = originalOverflow;
});

but it's not great, because the content is still getting clipped on firefox (notice the absence of right border).

chrome
firefox


Add Remuxing Support to MediaServer

I just decided to tinker around with the proposed solution in the thread below. I was able to implement MPEGParser for use on the MediaServer last night. I also think I was able to overcome the random crashing mentioned in the thread and so far, I haven't seen any files being stuck until I restart SageTV.  I able to make SageTV crash sometimes by closing the Remuxer twice, so the solution there was to just make it null once it has been closed. I also made it crash by overloading it with too much data at one time, which I solved by forcing the data to be fed in smaller chunks.
http://forums.sagetv.com/forums/showthread.php?t=62406
I was really impressed at how easy it was to get working. The bits I'm working on right now are just optimizing.
I noticed that MPEG-TS output only worked correctly when I fed it data in 188 byte increments. If I used anything bigger, the resulting video is playable, but corrupted. MPEG-PS output only complained when I went a lot higher than 64KB (around 2MB). Do these limitations sound familiar?
For setup, I used the suggestion of "REMUX_SETUP Format Channel InitSize." I implemented retaining the data used for detection, then once detection is successful, the data is written feed into pushData in 188 byte or 64KB chunks per the reasons in the above paragraph. I also added a check that if more than 5 seconds of time according to the container passed and the format has been detected to continue anyway even if we have not reached InitSize. That way you don't get stuck for a long time on low bitrate streams. Also if the InitSize is too little for the format to be detected, it will double in size up to 5MB before it will discard the buffered data and continue to buffer again. That should be way more than enough data based on my observations. After successful detection, the buffer is replaced with a 64KB buffer since that's the largest chunk it could possibly need to deal with when copying to the remuxer.
I was wondering about any suggestions on feedback to the network encoder so it can know when initialization is done and remuxing has actually started. This would be helpful to provide more seamless channel changes for example. Could I just add another command that allows the network encoder to query the current status of the remuxer?
The only downside I see to using the remuxing like this is if for some reason the network connection is disrupted. The remuxer doesn't appear to use any kind of offset, so it's unclear if you could actually add an offset as an option.

Groovy, nice work. :)  I'm not surprised at all by crashes on double close or requiring 188 byte alignment for incoming TS data (we have that alignment requirement on playback on the extenders as well). But, I would expect it to work with larger buffer sizes as well as long as they are multiples of 188. Feeding it 188 bytes at a time would be highly inefficient, so if you're seeing problems with sizes that are multiples of 188 as well, let me know and I can try to figure out what's going wrong.
As for the large buffers...there's likely some max size we are allocating buffers at and 2MB sound like a reasonable number we would have used.  If you want to know exactly, let me know and I can dig into that code to figure out what it is.
As for the feedback...you are going to want something in this area. Maybe add a different variant of the WRITE call such as REMUX_WRITE which then also returns the current size of the outputted data; and maybe even a status for whether the format has been detected yet (if you find that useful). The most important thing here is that when it calls GET_FILE_SIZE over the network encoder RPC you'll want to be sure it's returning the size of the remuxed output file that's been written out to disk and NOT the size of the data it's sent into the remuxer.  With this feedback, you would then know that size and can respond correctly.
And I wouldn't worry much about the network connection getting dropped; that pretty much never happens on LANs nowadays. And even if it does, SageTV's halt detector will kick in because the recording file has stopped growing and then it'll end up stopping and restarting the capture device in an attempt to recover (and it would likely recover then if the drop was due to a network anomaly).

I did try 1880 thinking similarly and that caused the TS remuxed output to be corrupted too. After far as I could determine, the limitation wasn't in the first method called in native code. I think it might be a little deeper. I thought it was interesting that PS is more tolerant. I also just in case I was somehow running on more than one thread unknowingly tried synchronizing the method; that did nothing helpful. :)

You don't need to dig into the code unless you're actually curious. I agree that 188 is rather inefficient, but I can live with it for now. It didn't hinder the file being written quickly enough.

Wait...I need to clarify something with the 188 byte thing. Were you saying that if you had it configured for MPEG2-TS output and then we also feeding it MPEG2-TS input...that you then needed to only give it 188 byte blocks in that case? I misread this before as when you gave it MPEG2-TS input it required 188 byte block inputs (regardless of output format).
If the input is already MPEG2-TS; then you really shouldn't even need to do the remuxing (although it then can also fix PTS issues relating to ad insertion, so I guess there is a benefit to remuxing the TS into TS again in that case). SageTV always defaulted to MPEG2-PS output; so if you just plan on having MPEG2-PS as your default output...and then if somebody decides to change that to MPEG2-TS, it then needs to do 188 byte blocks...that would sound fine to me then. :)  Defaults should always work the best...and if you want to go mucking around and changing things, then don't expect the changes to give you the best performance has always been fair in my mind.

I was testing MPEG2-TS to MPEG2-TS (only allowed 188-bytes at a time) and MPEG2-TS to MPEG2-PS. I have it set to use 188 bytes for TS output and 64K for PS output. I wouldn't actually mind being able to go back to PS; it always seems to work best with Comskip. :)
Many users are having problems with their cable providers inserting local commercials that badly disrupt the timeline, so remuxing is a must for them. 95% of the time I don't need to remux anything for my own setup, but there's that 5% of the time when I end up having to remux the raw stream or SageTV can't play it back. Also some Comcast content will not play at all in many situations without remuxing as it comes in because they do some strange things with the the order of the decode timestamps. So I think that even if it's TS to TS, it's worth it. I know you touched some of these points in your own comment.

Instead of waiting on feedback from a call to REMUX_WRITE (I kind of like the async nature of WRITE), I decided it would be more flexible to call what I've named REMUX_STATUS. It currently only accepts INIT as a parameter, but this leaves some flexibility in there in case something is thought of in the future.
I noticed that the SIZE call will do well enough. Based on the frequency of transfers being higher than the queries for the size of the file, I think it would be more efficient to just call SIZE when it's actually requested of the network encoder instead of returning the current size for each write when there's nothing to really useful to do with that information. I'm getting the current file size from fileChannel. Let me know if there's a more efficient way to get that information when remuxing.

Also, I was looking around trying to figure out how you typically do SWITCH to see if I can get that feature in there too. It looks like SageTV is able to start the new file, but I don't think it's actually restarting the muxer to do it. Do you have any hints about where I should really be looking?

I just added REMUX_SWITCH which changes the file out after flushing the last output from the remuxer. That way the remuxer doesn't need to be set up again and the transitions are very smooth. It takes the same format at WRITEOPEN and does the same checks. The only difference is that when it closes the file, it doesn't also close the remuxer. I have tested this approach on several different content types and it works very nicely. :) Basically I replaced allowing FileOutputStream with an anonymous OutputStream implementation that buffers the output of the remuxer and writes the buffered data out to whatever the current FileOutputStream is. On the OpenDCT side of things, I am selecting what looks like a good spot to split the stream, then it calls REMUX_SWITCH.

A few things to reply to here. :)
For the file size status; I agree about the nicety of the asynchronous part of WRITE. This could be dealt with even when it does replies of the size by having a multi-threaded receiver on the network encoder end where you process the replies in parallel...however, having written one of these for the way we send data to the extender, it's easy to make mistakes and have it break and get out of sync...so resorting to using SIZE when its needed isn't all that bad.  I'm not sure I followed what you were talking about wrt REMUX_STATUS.
What's probably the most ideal way to deal with the file size issue is to have another property setting which indicates that the SageTV server itself should get the file size. Then in NetworkCaptureDevice.java you just have it check the local file size there so that won't even then require an RPC to the network encoder (and then potentially another one back to the SageTV server just to make that same call).
I do see the point about remuxing TS back into TS; so you do want to support that.
For the SWITCH operation; that should be something you can handle on your end...the way we've always done it is to break the stream at either an I Frame (for MPEG2) or an IDR (for H264). So if you detect that case and then switch it at that point, you should be fine. Remember, you should not return from the SWITCH call until you have actually completed the SWITCH operation (which means its OK to block in that for a very short amount of time, generally it'll take up to a half second to detect the right boundary due to 0.5 second GOPs).
And I would expect the remuxer to strip the PAT/PMT from the TS stream because it's moving them to different PIDs so it can't retain those from before. I would expect it to write out new ones though; but I don't recall if it actually does that (I didn't write that code).  Although I took a quick look at it, and it appears that it will write out a PAT every 2000 packets and a PMT every 8000 packets. So if you have remuxed files where MediaInfo is complaining there's no PAT/PMT, send me the first couple MBs of the TS file and I'll take a look at it. (but from the code the first packet TS packet in the file should be PAT and the second one PMT).
As for H264...I don't think I've ever seen that in an MPEG2-PS...so I'd say stick with TS for any H264 channels.
Hopefully I got all your questions...let me know if you have more. :)

Thank you for so much help. You've been awesome. :)
This is my experience with TS as input and PS as output. I learned that switching at an I frame on the input stream wasn't good enough. What was happening when I was watching live TV is I would switch at an I frame (starting the new file with the detected I frame) and the player would lock up while the new file continued to be written. If I pushed the I frame just before switching the file, it would transition, but the audio would drop out, then if I skipped back a little in the live recording, the audio would come back. I learned early in developing OpenDCT not to return OK until everything was ready. :) These results are why I changed my focus to the output of the remuxer, but if you're saying I should be able to do this on the input side, I'll try to figure out what isn't doing what I think it's doing.
In OpenDCT, when I SWITCH while remuxing with FFmpeg, I can do it on any key video packet and it always works really well whether I'm using TS or PS. I also have switching working well with the raw output option too, so this is a little perplexing to me.
As far as I could determine, it didn't actually change the PID's. I scanned a 3.7GB file it created with one of the functions I wrote in OpenDCT and it contained absolutely no packets with the PID 0 (PAT). I didn't have a function for finding a loose PMT, but I have a feeling wasn't there either. How would you prefer I get you a sample; also do you want the unprocessed file, the processed file or both?

So I'm not sure at all that you can do this on the input side or not. My assumption would be that you could..however, that assumption may be wrong. :)  The remuxer is likely buffering the input data to some degree in order to keep the PS packet sizes somewhat consistent. However, I would also assume it employs some kind of I-Frame detection and that it would start a new PS packet when it encounters an I-Frame since that is generally desirable.
I took a quick look at the code...and I think it's doing some kind of alignment, but I'm not sure.
As for not changing the PIDs...I figured it would...but I'm not too surprised if it isn't doing that. :)
For my own analysis, I'd like the input and output files you have. Just upload them to Google drive and make it shareable and post the link here so I can download them and take a look.

I made a little progress over lunch today, so I want to see if I can work this one out myself. I really need to write more C++ (for something other than a microcontroller) and I feel like this is a good opportunity to learn something. I like a good challenge and I think I figured out why there are alignment issues in the native code when feeding TS packets in sizes larger than 188.

Great to hear it. :)  I don't know how deep your knowledge of MPEG standards is; but if you need any help understanding why certain things are being done at that level, always feel free to ask...I've written far too many MPEG parsers. :)

I'm not an expert by any means, but I will say I know a lot more about how it all works than I did a year ago. :) In this case I have the advantage of the code in place already being fairly solid, so all I'm really looking for is the quirk that's making it not work as expected.

From what I am able to determine, BlockBufferTSDump (TSBuilder.c) is the only method that uses PushTSBlockData (TSBuilder.c) and BlockBufferTSDump is a callback set by OpenRemuxStream (Remux.c). So if we are looking at the same file, basically those PAT and PMT packets will never be injected, or at least not for the reasons you would think. Just to verify a few things I set up some additional logging in these methods and as expected, the logging never happened. The method that's actually being used is ParseData (TSParser.c). I verified that the packets are in fact taking this route, but when the packet is a PAT or PMT packet, it is processed, then discarded. I see a path to getting this working as expected, but after seeing what route the packets are taking, I'm not even sure discontinuities are being corrected.

I figured out how to get the PAT and PMT packets injected. I was following the code all the way to TSAVStreamHook (TSSplitter.c) which is the callback that ultimately queues the packets for writing and also happens to keep a count of how many it has written. I didn't see anywhere else that updated this counter, so I figured it was safe to use for intervals. I made a few changes to that method and suddenly PAT and PMT packets at the right intervals started being injected into the TS output. PushPat and PushPmt looked like they were only in use at the end of some code for detecting an ATSC channel.
//start build video or audio from start frame (PES header)
if ( ts->builder_state >= 2 )
{
	int packet_increment = 1;

	// Insert PAT and PMT packets at interval.
	if (ts->ts_packet_num % 2000 == 0)
	{
		PushPat( ts->ts_builder );
		packet_increment++;
	}

	if (ts->ts_packet_num % 8000 == 0)
	{
		PushPmt( ts->ts_builder, ts->program_id );
		packet_increment++;
	}

	PushAVPacketData( ts->ts_builder, ts->program_id, av_info->sub_channel,
			  av_info->start_group_flag, av_info->data, av_info->bytes );

	ts->ts_packet_num += packet_increment;
}

Groovy, nice detective work. :)

I noticed that on TS output PCR packets are being dropped. I think this might be the cause of some audio drift I was seeing while watching live TV. There certainly are a lot of little issues with this code path. :)

PCR shouldn't affect anything. I've never seen a modern decoder that even
uses it. Audio/video sync should be fully maintained by the PTS values. The
only time I ever see drift anymore is due to underflow...but we prevent
that from occurring in the extenders. If you have a repro case, I'd be more
than happy to look at it.
Jeff Kardatzke
Sent from my Android
On May 18, 2016 6:06 PM, "Joseph Shuttlesworth"<EMAIL_ADDRESS>wrote:

I noticed that on TS output PCR packets are being dropped. I think this
might be the cause of some audio drift I was seeing while watching live TV.
There certainly are a lot of little issues with this code path. :)
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-220201677

The live TV situation was with the SageTV client. I didn't do much testing with the extender. Also, while I can say that while SageTV can decode the streams without any real issues, Windows Media Player (which can play the raw stream surprisingly) and VLC cannot. FFmpeg doesn't have any complaints about the files either. I guess your comment about a modern decoder might be the key word. I found the location where it appears to be discarding all of the packets containing PCR data because it can't parse them correctly and think it would be better to fix it. Some people other than myself prefer that the files are able to playback in places other than SageTV. I'll let you know if I get stuck.

Not sure if this is related, but I've had repeated similar cases with playback on HD200's where the audio mysteriously fades out for 1-2 seconds and then fades back in.  Occurs on multiple systems running Sage 7.1.9, but not using OpenDCT.  Although it most frequently occurs while watching live TV (the nightly news), it's also occurred during playback of Sage-recorded TV series.  In every case, skipping back and re-playing the segment repeats the same audio problem.   These are .mpeg files that were recorded OTA from an HDHR tuner.  If it'll help, I can probably provide an affected file.

@JustFred51 These issues I'm talking about right now are just with TS output from the remuxer built into SageTV. Typically no one really uses this code path and I'm trying to shape it up so H.264 content can be fed through it reliably. My plans are to start with PS, then if the video isn't MPEG-1 or MPEG-2, switch over to TS. The PS (.mpg) output appears to be ok.
@Narflex You're right (as I'm sure you expected :)) about the extender. The audio isn't drifting at all.

Unrelated, but I did figure out why 188 per transfer was required. The issue was that the packet queue only had enough play for one additional packet. I bumped that value up 7 more and now I can push 7 packets at a time which is a typical number of packets for RTP, so I think that's a good number to work with.

I confirmed that it's not that it's dropping the PCR packets so much as it's wiping the PCR data out and clearing the adaptation field PCR flag. The culprit is in BuildTSHeader (TSBuilder.c) which basically wipes out all of the adaptation data. I suppose this would not be a problem since this data isn't used for anything as far as SageTV is concerned.

What's is the code under ./native/ax/Native2.0/NativeCore/TSParser.c? It looks like a more complete implementation. The code that I have been working with is under ./native/ax/TSnative/TSParser.c.

I didn't even realize there was still code under native/ax/TSnative...apparently that's the old school remuxer/TS handling code. The stuff in ax/Native2.0/NativeCore is the updated version that was done to handle more stuff.  We barely used the remuxer from the Java level at all (it was only for the MediaMVP)...and now I recall telling Qian that we should just leave the original code for that in there for compatibility,  and he didn't need to worry about merging them together (many years ago).
So...ideally they would all get merged together and the Java remuxer would use the Native2.0 code; but that may be a much larger undertaking and likely non-trivial (otherwise we probably would have done it).  For the remuxing done by our own capture code...I know it is using Native2.0.
@qianzhang5 Any comments on this?

I had a bad feeling that you were about to essentially tell me I've wasted 3 days looking at the wrong code.

I wouldn't say you've wasted three days...there should likely be lots of overlap in the structure of the two parts, especially the higher level remuxing that's done.

I actually was trying to find a silver lining in this. I think from looking at how the simpler version works, I will have an easier time following what's going on in the more complete version as I port it over to being able to be used in Java.

I finally got around to taking a closer look at the feasibility of adding use of the newer code. I was originally thinking I could just upgrade the current native code that uses the old remuxer to use the newer remuxer, but it looks like there are places in SageTV that still use the old remuxer code to determine container formats.
I guess I will need to create a new .dll for Windows, a new .so for Linux and a new Java class to pull the JNI in.

That's interesting..I didn't realize it was using the old code for format detection still...but I guess that makes sense since they are in the same library.

So if I'm looking for a rough guide on how to use this code correctly, ./native/ax/Native2.0/RemuxFile/RemuxFile.c is the right place to be looking? Or do you have a better suggestion?

Yeah, that should lead you down the right path. That'll point to native/ax/Native2.0/NativeCore/Remuxer.c which then has more high level remuxing code in it. You'll need to use something different than the file based one; but by looking at that file you should be able to figure out how to set it up for something that's memory based instead of file based.

I was really confused for a little while until I noticed that in ./native/ax/Native2.0/NativeCore/NativeCore.h, there is a macro that reads #define NULL    ((void *)0). NativeCore is entirely done in C. Leaving this macro intact from what I understand isn't valid in C++ and prevents any possibility of compiling a Windows DLL. I added
#ifdef NULL
#define NULL 0
#endif

to the header for the DLL and it fixed the issue, so it will actually compile now, but I'm not sure what consequences this might have down the road. Is there a reason why NULL was defined? If we need to be positive of the value, could we possibly use a less global macro?

I can't remember the reason now. It probably was for cross-platform
compatibility in embedded c in the early days. I don't think there is a
risk here.
On Fri, Jun 3, 2016 at 10:06 PM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

I was really confused for a little while until I noticed that in
./native/ax/Native2.0/NativeCore/NativeCore.h, there is a macro that reads
#define NULL ((void *)0). NativeCore is entirely done in C. Leaving this
macro intact from what I understand isn't valid in C++ and prevents any
possibility of compiling a Windows DLL. I added
#ifdef NULL
#define NULL 0
#endif
to the header for the DLL and it fixed the issue, so it will actually
compile now, but I'm not sure what consequences this might have down the
road. Is there a reason why NULL was defined? If we need to be positive of
the value, could we possibly use a less global macro?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-223736901, or mute
the thread
https://github.com/notifications/unsubscribe/AMQtgE1do4gn7nOHheBRqNzGAylfHiAPks5qIQfjgaJpZM4IeL_U
.

Now that I am able to create and close the newer remuxer in the JVM, I moved on to pushing data into it. I noticed that what comes out is actually messages and data in a struct. I see an example of how to deal with all of the messages in /third_party/Microsoft/TSSplitter/TSSplitFilter.cpp. It's a fair amount of code to just copy, so I was wondering what the preference would be here or if a lot of that code isn't even needed to detect and remux the stream.

Preference between which options?
Jeff Kardatzke
Sent from my Android
On Jun 6, 2016 7:10 PM, "Joseph Shuttlesworth"<EMAIL_ADDRESS>wrote:

Now that I am able to create and close the newer remuxer in the JVM, I
moved on to pushing data into it. I noticed that what comes out is actually
messages and data in a struct. I see an example of how to deal with all of
the messages in /third_party/Microsoft/TSSplitter/TSSplitFilter.cpp. It's a
fair amount of code to just copy, so I was wondering what the preference
would be here or if a lot of that code isn't even needed to detect and
remux the stream.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-224116200, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDCw2TlFetHcw0uiXBfdWUakt1Baiks5qJKjIgaJpZM4IeL_U
.

I could have worded that better. Should I just copy it verbatim with minor adjustments, are there any suggestions on what parts really shouldn't be needed or would you rather I just use my best judgement?

Use your best judgment, unless Qian chimes in with advice
Jeff Kardatzke
Sent from my Android
On Jun 6, 2016 8:20 PM, "Joseph Shuttlesworth"<EMAIL_ADDRESS>wrote:

I could have worded that better. Should I just copy it verbatim with minor
adjustments, are there any suggestions on what parts really shouldn't be
needed or would you rather I just use my best judgement?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-224127375, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDEq8jrRuPsatqGk7elJBQ6tVQ-Mkks5qJLkxgaJpZM4IeL_U
.

I think I was looking at this a little too intensely. Getting remuxed data out was a lot easier than I was making it out to be. Very exciting. :)

Awesome!
Jeff Kardatzke
Sent from my Android
On Jun 6, 2016 10:08 PM, "Joseph Shuttlesworth"<EMAIL_ADDRESS>wrote:

I think I was looking at this a little too intensely. Getting remuxed data
out was a lot easier than I was making it out to be. Very exciting. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-224144300, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDILKoBwVR9COcf6tSQQ2lFT7RFy6ks5qJNKAgaJpZM4IeL_U
.

One of the things I think would be desirable is that is if the stream does not have MPEG-1 or MPEG-2 video that the container should automatically be changed over to TS if the remuxer was going to output PS. I found a function in Remuxer.c that would make you think it would be able to do this based on it's name (ChangeRemuxOutputFormat), but if I call that after the format has been detected, nothing is written. I can see that function migrates the DUMP callback and associated context, so I think this issue has more to do with changing the format at that point is simply wiping the detected stream info, causing the remuxer to essentially drop everything because it doesn't know what you want.
The workaround that I am using right now is to just re-open the remuxer setup to output TS which I guess is ok since I'm buffering the data until the format is detected, then flushing the remuxer queue and pushing the data in from the start (which nets me a few extra seconds of video). Since detection is more of an inline function with the newer remuxer, I had to be a little more clever so I wouldn't essentially dump some of the stream during detection which is what it appears to do with BDA devices. I know that having those few extra seconds buffered makes playback start faster, so this buffer during detection is very desirable.

I got it all working within the framework that I built for the older remuxer. All I have left to work out now is getting all of the switching correct and I'll finally have something to commit! Of course that might be another week or two being this doesn't really pay the bills. :) I'd like to support more TS transitions than one specific to the HD-PVR, so that may take a little longer. I'm going do all of that in Java since if it can be improved it will be more likely that someone might step up.

I ran into a few channels that have 3 audio tracks that are showing up as audio only. I'm having a hard time figuring out how to send a hint to detection that I need at least one video and one audio track. Any suggestions on where I should look to fix this?

I'm not sure I understand exactly what you mean....but are you saying that
it's not detecting the video stream on some channels? If so, do you have
the Native.log from when that occurs? (I think I know what this one is and
can help you resolve it)
Jeff Kardatzke
Sent from my Android
On Jun 8, 2016 10:21 AM, "Joseph Shuttlesworth"<EMAIL_ADDRESS>wrote:
I ran into a few channels that have 3 audio tracks that are showing up as
audio only. I'm having a hard time figuring out how to send a hint to
detection that I need at least one video and one audio track. Any
suggestions on where I should look to fix this?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-224604607, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDOIBEb-sIqAfeZXqTLI9Z1rugQ_4ks5qJs_9gaJpZM4IeL_U
.

Yes, the video is not being detected on some channels. Actually I just noticed that one of the audio tracks is missing on one of them too. The attached log for a TS that has 1 video and 3 audio streams.
Native.log.txt

That's not the issue I was thinking of. Can you post a small TS file that
can reproduce the issue with SageTV format detection? Be aware I also made
a change recently which will use FFMPEG in this case to workaround a
problem relating to this.
That also reminds me....when I was looking at that other problem before, I
swore I was editing code in native/ax/Native2.0 for debugging the format
detector...I think the Java lib may only use the old code for remuxing, but
the new code for format detection.
Jeff Kardatzke
Sent from my Android
On Jun 8, 2016 11:13 AM, "Joseph Shuttlesworth"<EMAIL_ADDRESS>wrote:

Yes, the video is not being detected on some channels. Actually I just
noticed that one of the audio tracks is missing on one of them too. The
attached log for a TS that has 1 video and 3 audio streams.
Native.log.txt
https://github.com/google/sagetv/files/305079/Native.log.txt
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-224621207, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDAxagPIqjwGzlTM6JEdA1Dhp3NM9ks5qJtv9gaJpZM4IeL_U
.

That's entirely possible. I didn't want to break anything that's not already broken in my adventures, so I figured it was safer to er on the side of caution when I noticed for example that the wizard was using it for example.
Here's a link to the file I ran through FileRemux.exe; it's two minutes:
https://drive.google.com/open?id=0B4AD2tiek6Xic0N0di00SU9tMTA

OK, cool...the first Native.log you posted didn't have what I was looking for (it appears there were only audio streams in that one). But the file you posted, and the output from RemuxFile that you posted both have what I wanted to see (i.e., I understand the problem).
There's this part in there for the video stream:
Type:0x80 Pid:0xbc3
The 0x80 type is DC-II, and it's been a problematic one. That type isn't really definitive overall, as we've seen it be non-video in imported files before. However, there's logic in the format detection code where if it has a 'hint' that it's analyzing a TV source, then it will assume that the 0x80 type is for video and then the format detection should come out correct.
The fix for you shouldn't be that bad to do at all. I'm not sure where you want to put it because I don't know where exactly you are tying into the remuxer...but I'll walk you through the origin of it so you can decide where to fix it.
The fix lies in ax/Native2.0/NativeCore/TSFilter.c in the conditional on line 1510. The stream type is 0x80, so it's passing those parts...but the DCII and Video Desc tags are not in the stream. So you're relying on the inner part of the conditional where it checks for 0x80 type and that IsTVMedia(nMediaType) is true.  So the way to fix it is to ensure that IsTVMedia(nMediaType) is true (and it's safe to do that, because you are analyzing a TV recording).
SelectTSFilterChannel gets called from TSParser.c in various places, and it always takes that media type from pTSParser->media_type. There's a couple ways you could get that set...if the TUNE parameter passed through everywhere is set properly, then the media_type will get set to TV in OpenTSChannel.  Alternatively, you can set the media type directly in the TSParser pointer in the DEMUXER struct.  At a higher level, I'd recommend passing a boolean down from Java that specifies whether or not the remuxer is analyzing a TV Recording (which is easy to determine from MMC.getInstance().isRecording(java.io.File)).
Hopefully that'll fix it all right up for you. :)  If you need more info, just let me know.

Fantastic. After I followed the code you were referring to, suddenly everything made sense. I was able to configure the TUNE parameter to get the desired result. I had a feeling that was key to get this to work, but I had a lot of trouble figuring out what needed to be set to make it realize that I was tuning TV. Thanks for the great assistance. :)

Glad you got it working. :)  I'm always here to help.
On Thu, Jun 9, 2016 at 1:36 PM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

Fantastic. After I followed the code you were referring to, suddenly
everything made sense. I was able to configure the TUNE parameter to get
the desired result. I had a feeling that was key to get this to work, but I
had a lot of trouble figuring out what needed to be set to make it realize
that I was tuning TV. Thanks for the great assistance. :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225018324, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDCRY5kumdjxTV3VIY-HuXIWjxUPzks5qKHk2gaJpZM4IeL_U
.

--
Jeffrey Kardatzke
<EMAIL_ADDRESS>Google, Inc.

In the UNKNOWN_CHANNEL struct, is tsid equivalent to the program number or am I missing it's purpose completely? I noticed that when tune_string_type in the TUNE struct is set to 1 (program) that ConsolidateTuneParam() still uses the value of unkn.data1, but I don't think that's actually being used as the program number. I hope this isn't too confusing. I'm trying to make as much of this as is reasonable available to the JVM.

I'm not sure exactly what's going on there...I didn't write that code and it's fairly complex so I'm not sure how that value is getting used in other areas (grep finds a lot of hits).  Since you're only doing the remuxing after the proper TS should have already been extracted; you probably aren't going to gain much functionality wise from dealing with the TUNE struct. Personally, I'd just go with the flag for forcing it to the TV media type and not bother to make it more advanced than that....unless you already have use cases that would matter for that....then by all means go for it. :)

If there is only one program (one channel) in a file there should not be a
problem using this code to remux. if there are multiple channels in data,
picking up a correct channel replies on TUNE setting which needs program
id, tsid or other parameter. because the code underneath handles ATSC, QAM,
DVB-T/C/S standards, there is not one simple way to work for them all. it
may confuse.
On Fri, Jun 10, 2016 at 7:19 AM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

In the UNKNOWN_CHANNEL struct, is tsid equivalent to the program number or
am I missing it's purpose completely? I noticed that when tune_string_type
in the TUNE struct is set to 1 (program) that ConsolidateTuneParam() still
uses the value of unkn.data1, but I don't think that's actually being used
as the program number. I hope this isn't too confusing. I'm trying to make
as much of this as is reasonable available to the JVM.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225194673, or mute
the thread
https://github.com/notifications/unsubscribe/AMQtgFwfd-LuJTcw7x199jx42xe3odm8ks5qKXJ9gaJpZM4IeL_U
.

Understood. Usually the driver or hardware is filtering out the other programs on the same frequency. I was just trying to cover as much ground as is reasonable. I exposed what I'm reasonably certain I understand to the JVM. The code allows the person using the remuxer to decide just how deep they want to go based on what method they use to open the remuxer.
From the MediaServer perspective, the command is now REMUX_SETUP mode outputFormat parameters.... The only mode supported right now is AUTO and other modes could be added if they are needed for whatever reasons. The format for AUTO would be REMUX_SETUP AUTO outputFormat isTV. Ex. REMUX_SETUP AUTO PS TRUE

Sounds good to me. :)
On Fri, Jun 10, 2016 at 11:40 AM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

Understood. Usually the driver or hardware is filtering out the other
programs on the same frequency. I was just trying to cover as much ground
as is reasonable. I exposed what I'm reasonably certain I understand to the
JVM. The code allows the person using the remuxer to decide just how deep
they want to go based on what method they use to open the remuxer.
From the MediaServer perspective, the command is now REMUX_SETUP mode
outputFormat parameters.... The only mode supported right now is AUTO and
other modes could be added if they are needed for whatever reasons. The
format for AUTO would be REMUX_SETUP AUTO outputFormat isTV. Ex. REMUX_SETUP
AUTO PS TRUE
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225262952, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDLkKcgnrBqBMFrhwPJh3mGyfkLhtks5qKa-vgaJpZM4IeL_U
.

--
Jeffrey Kardatzke
<EMAIL_ADDRESS>Google, Inc.

So I ran into something unexpected trying to find an I frame in some HD MPEG-2 TS files. It looks like it's only interlaced content. I can find the 4 byte header 0x00000100, but the only picture codes I'm getting are B and P, no I. Have you ever seen this before? Is this why MulticastCaptureDevice doesn't try to find a transition point for MPEG-2 files?

No, I haven't. you may use ffprobe with -show_frames to check.
On Sun, Jun 12, 2016 at 6:43 PM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

So I ran into something unexpected trying to find an I frame in some HD
MPEG-2 TS files. It looks like it's only interlaced content. I can find the
4 byte header 0x00000100, but the only picture codes I'm getting are B and
P, no I. Have you ever seen this before? Is this why MulticastCaptureDevice
doesn't try to find a transition point for MPEG-2 files?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225473873, or mute
the thread
https://github.com/notifications/unsubscribe/AMQtgHjKS5S6EQYVrKVWRU-9AIoJHuNmks5qLLWtgaJpZM4IeL_U
.

I haven't seen that problem before either, and the Java code didn't do it
because we just never wrote it.
Jeff Kardatzke
Sent from my Android
On Jun 12, 2016 10:53 PM, "qianzhang5"<EMAIL_ADDRESS>wrote:

No, I haven't. you may use ffprobe with -show_frames to check.
On Sun, Jun 12, 2016 at 6:43 PM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

So I ran into something unexpected trying to find an I frame in some HD
MPEG-2 TS files. It looks like it's only interlaced content. I can find
the
4 byte header 0x00000100, but the only picture codes I'm getting are B
and
P, no I. Have you ever seen this before? Is this why
MulticastCaptureDevice
doesn't try to find a transition point for MPEG-2 files?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225473873,
or mute
the thread
<
https://github.com/notifications/unsubscribe/AMQtgHjKS5S6EQYVrKVWRU-9AIoJHuNmks5qLLWtgaJpZM4IeL_U
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225494536, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDLTBkNBHQixMOZ5CF1l_aLKbMc4uks5qLPBUgaJpZM4IeL_U
.

@qianzhang5 Thanks for the suggestion. ffprobe found them and pointed me right at the offset I should be looking at. I was also able see what I was looking for in a hex editor.
I made a really dumb mistake in reading the test file...basically it wasn't reading new data and the I frame was just past the length of the buffer.  Sorry for the hassle and thanks for the quick replies.

Are we aware that building libNativeCore.so with gcc 5.3.1 may result in what appears to be data alignment issues? I was able to compile a good copy with gcc 4.8.4. I thought I messed something up that simply didn't appear in Windows. I'm glad I tried compiling on Ubuntu 14.04. The resultant library from the newer compiler was writing out files with small amounts of seemingly random corruption.

Make sure your in new code variables are portable (such as int8_t,
int32_t...) and test code with 64 bit and 32 bits compiler on windows and
linux.
On Mon, Jun 13, 2016 at 8:49 PM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

Are we aware that building libNativeCore.so with gcc 5.3.1 may result in
what appears to be data alignment issues? I was able to compile a good copy
with gcc 4.8.4. I thought I messed something up that simply didn't appear
in Windows. I'm glad I tried compiling on Ubuntu 14.04. The resultant
library from the newer compiler was writing out files with small amounts of
seemingly random corruption.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225772937, or mute
the thread
https://github.com/notifications/unsubscribe/AMQtgM5RizwFc4y9WO3tpxRbg0Hzzy0Qks5qLiTEgaJpZM4IeL_U
.

@joseph - is the gcc4.8.4 still 64bit?  It is interesting that 5.3.1 would
cause corruption, specially if they both 64bit.  My docker container for
building sagetv is ubuntu14..04... but that was mainly because I can't
build mplayer with anything newer.  At some point I'd like to use a newer
gcc.
On Tue, 14 Jun 2016 at 01:52 qianzhang5<EMAIL_ADDRESS>wrote:

Make sure your in new code variables are portable (such as int8_t,
int32_t...) and test code with 64 bit and 32 bits compiler on windows and
linux.
On Mon, Jun 13, 2016 at 8:49 PM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

Are we aware that building libNativeCore.so with gcc 5.3.1 may result in
what appears to be data alignment issues? I was able to compile a good
copy
with gcc 4.8.4. I thought I messed something up that simply didn't appear
in Windows. I'm glad I tried compiling on Ubuntu 14.04. The resultant
library from the newer compiler was writing out files with small amounts
of
seemingly random corruption.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225772937,
or mute
the thread
<
https://github.com/notifications/unsubscribe/AMQtgM5RizwFc4y9WO3tpxRbg0Hzzy0Qks5qLiTEgaJpZM4IeL_U
.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225786385, or mute
the thread
https://github.com/notifications/unsubscribe/ABAfKW065fY3xNz4kSj-DvITjSe8PHtjks5qLkG2gaJpZM4IeL_U
.

--
Sent from Android

For the most part I have matched the variables to exactly how they are defined within the existing SageTV libraries. I will change the code that I'm adding to use the platform independent variables. The signatures in the core libraries are often using int and unsigned long, so I will not be changing those.
I know the latest gcc compiler doesn't play nicely with several pieces of software that work fine on the older one. I don't know what the changes are that might be breaking them, but I've mostly seen it break A/V software. It causes the software to think something is wrong with the packets and it tries to fix them which just results in them being mangled or some of the original content being wiped. To me that sounds like somewhere we are using a variable that might be involved in some bit-math and the number of bytes in the variable in this compiler have changed.

@stuckless Yes, both compilers are 64-bit. The testing of the compiled libraries was done on a Fedora 23 64-bit VM. It's worth noting that SageTV started acting a little strangely with the libNativeCore.so compiled with the newer gcc. When I re-compiled everything with the older version of gcc, all was well again.

What would it take to have an option for the remuxer to output other containers as well (not just m2ps or m2ts, but mkv maybe)?

Qian should have a better idea on this, but based on what I've seen, someone would need to add another path for the incoming packets to travel to go into an MKV container. Someone would also need to add code to the push code on the Java side of things so the video playback would not get too far ahead or too far behind the live feed. If you're not arguing for live playback, then FFmpeg is the better route since that means no one on the SageTV side of things will have much to maintain.

When you were doing the compiler comparisons, did you do a full rebuild of
everything rather than just the one library? It may be worth checking
another tree out just to be sure it is completely clean, although I don't
believe there are any artifacts left behind after doing a clean that would
cause problems....but it's possible.
Jeff Kardatzke
Sent from my Android
On Jun 14, 2016 8:54 AM, "Joseph Shuttlesworth"<EMAIL_ADDRESS>wrote:

Qian should have a better idea on this, but based on what I've seen,
someone would need to add another path for the incoming packets to travel
to go into an MKV container. Someone would also need to add code to the
push code on the Java side of things so the video playback would not get
too far ahead or too far behind the live feed. If you're not arguing for
live playback, then FFmpeg is the better route since that means no one on
the SageTV side of things will have much to maintain.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-225927325, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDNi3nohOvNCO93ZB5lqfB1AJ_26kks5qLs6lgaJpZM4IeL_U
.

I did a full rebuild; then a few partial while I was trying to figure out what went wrong. Basically after I confirmed that I was able to compile everything successfully in Linux and Windows, I committed all of my changes to GitHub, then used git clone in a new location to be sure it would compile from scratch.

I was starting to work on integrating the switching and size checking so that it doesn't even hit the network encoder. It looks like channel changing used to also be a part of switching, but we stopped doing that. Is that no longer the case or do I need to account for this and pass the switching on to the network encoder if the channel changes? I don't think I've ever seen the channel change at the same time as a switch.

We never change the channel the same time we do a SWITCH; that may have
been an artifact from a long time ago (because in the old days, SWITCH used
to mean something a little different).
On Thu, Jun 16, 2016 at 9:52 AM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

I was starting to work on integrating the switching and size checking so
that it doesn't even hit the network encoder. It looks like channel
changing used to also be a part of switching, but we stopped doing that. Is
that no longer the case or do I need to account for this and pass the
switching on to the network encoder if the channel changes? I don't think
I've ever seen the channel change at the same time as a switch.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-226545625, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDNRy5dT_Iv6-OWxWkxN6T9qVyjbAks5qMX9igaJpZM4IeL_U
.

--
Jeffrey Kardatzke
<EMAIL_ADDRESS>Google, Inc.

Got it. That means that SWITCH can happen without even telling the network encoder. That will be a very cool feature since once this is added, any network encoder will be able to support SWITCH by simply using media server remuxer.
The way I am tying the remuxer into the network capture device is via a ConcurrentHashMap by File. The value returned is a MediaServerRemuxer which is a new class I added so I wasn't cluttering up the MediaServer class. When a MediaServerRemuxer is closed, the mapping is removed from the HashMap. When switching, the old File is removed from the HashMap and the new File is added with the same object reference.
GET_FILE_SIZE is trivial with the HashMap and is working great.

Oh yeah, great point...that would remove the Network Encoder from having to
do anything for SWITCH if it's using your remuxer. :)
On Thu, Jun 16, 2016 at 11:42 AM, Joseph Shuttlesworth <
<EMAIL_ADDRESS>wrote:

Got it. That means that SWITCH can happen without even telling the network
encoder. That will be a very cool feature since once this is added, any
network encoder will be able to support SWITCH by simply using media server
remuxer.
The way I am tying the remuxer into the network capture device is via a
ConcurrentHashMap by File. The value returned is a MediaServerRemuxer which
is a new class I added so I wasn't cluttering up the MediaServer class.
When a MediaServerRemuxer is closed, the mapping is removed from the
HashMap. When switching, the old File is removed from the HashMap and the
new File is added with the same object reference.
GET_FILE_SIZE is trivial with the HashMap and is working great.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/sagetv/issues/138#issuecomment-226576112, or mute
the thread
https://github.com/notifications/unsubscribe/ANEIDKl-UtGnH4v1EH_a1WTWy31Vn0Ptks5qMZksgaJpZM4IeL_U
.

--
Jeffrey Kardatzke
<EMAIL_ADDRESS>Google, Inc.

I have been using the new remuxer for several days in production (Windows) and on my testing VM's (Windows & Linux 64-bit). It's working great. Let me know if you need me to make any changes. It really seemed like this would be a bigger task than it actually turned out to be.
#143

Glad to hear it wasn't as bad as you thought. :) SageTV has LOTS of stuff inside of it...so quite often things can get done much easier than you may think. :D
I reviewed the code; the Java code I went through in more depth...my main goal is to ensure you don't break anything that's there. :) I glossed over the rest of it; and it looked fine to me. I did have a few comments on the Java code.   I'm assuming you'll also have a flag in the network encoder code of yours to turn this on and off as another type of debugging assist if problems arise.
And thanks for taking this on! I greatly appreciate developers contributing to SageTV.  Also, don't forget to add your name to the CONTRIBUTORS file if you haven't done that already.

OpenDCT has several ways to stream the video, you can stream it raw directly to a file or via the media server, remuxed or transcoded via FFmpeg directly to a file or via the media server and I just added another method which uses the remuxer I just added to the media server. So if you're counting, that's 5 different ways to record.
I also added a command to disable the network encoder "assistance" that allows SageTV to talk directly to the remuxer instead of the network encoder for SWITCH and GET_FILE_SIZE in case something odd is discovered.
I really really like this software. It has so much already and I really want to see it thrive. If full-time development was an option, I'd be all over it. I added myself to CONTRIBUTORS.

75 comments later... Done! :)


[NF] relative paths + [BC] InCurrentDQ

added missing function of InCurrentDQ
material data from material library use relative path now

Thank you for your contribution!
Great idea to gather everything in definition. Maybe one day we can extend it to other global variables (Exceptions ? plot colors ?)


⚠ Bofrost.be has degraded performance

In 82c0347, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1646 ms

Resolved: Bofrost.be performance has improved in 8b6a9ac.


its not working

sir its not work perfectly undefined fftOneSide

fftOneSide this function file only not upload here if you want just contact my mail

fftOneSide this function file only not upload here if you want just contact my mail

how can i contact ur mail sir?


[REG 2.067] Issue 15490 - Error variable __nrvoretval cannot be modified at compile time when using -inline

CTFE interpretr can recognize a form CommaExp(DeclaraationExp(v), VarExp(v)) as a CTFEable variable declaration when v._init is null. In other cases, for examle a sole DeclarationExp should have v._init for CTFE-ability.
Supply VoidInitializer to allow inlining the temporary for NRVO.

Thanks for this.

Auto-merge toggled on

Thanks!


Jumps to top when menu opens

I followed the basic usage steps and I get this weird issue with the panel scroll position jumps to the top when the menu opens. I feel like it's something I'm doing to cause this but I can't figure out why it's jumping to the top of the content when you open the menu at a scroll position that is not at the top.
http://codepen.io/nstanard/pen/VaMbom

Is happening when the slideout-open class gets added... removed overflow: hidden; and it's not jumping to the top anymore...
.slideout-open,
.slideout-open body,
.slideout-open .slideout-panel {
/*   overflow: hidden; */
}

Height: 100%; on the html tag causes this.

I'm having the same issue and identified that style as one reason why. But removing this style results in weird touch behavior, like not being able to close the menu, or the panel/menu getting stuck halfway in between open/closed. I'd love to know why the panel is jumping to the top!

I had the exact same issue and solved it by removing only the ".slideout-open body" from the CSS mentioned above. Don't know if this still leads to weird behavior for you @bbatliner. In my case, the trigger button is placed outside of the .slideout-panel and I can open and close the menu without any problems.

I was able to fix the issue by leaving all of the styles, but changing other parts of my page so you could still scroll inside the child div, even if the  has overflow: hidden.

So, I decided to ditch slideout. Turns out CSS transitions were all I needed for a mobile menu. My solution now simply adds and removes my sso-mobile-nav-is-open class which causesthe CSS to slide the main content over to reveal a mobile menu. Super simple solution. I also use CSS to transition the visibility of the menu while its' open/closed.
`
/**
*
/
/ MAIN CONTENT CONTAINER (MOBILE NAV IS CLOSED) /
#sso-main-content {
background: $white;
transition: transform .3s linear 0s; // header specific
border-left: 1px solid #f1f1f1; // header specific
margin-left: -1px; // header specific
//margin-top: 60$px; // header specific
width: calc(100% + 1px); // header specific...
}
/**
 *
 */
/* MAIN CONTENT CONTAINER (MOBILE NAV IS OPEN) */
#sso-main-content.sso-mobile-nav-is-open {
    transform: translateX(260px); // 260PX IS THE MOBILE NAV SIZE
    transition: transform .5s linear 0s;
}

`


Added config property instructions to enable Blocking Module in PBS Java

Description
In the code for the oRTB Blocking Module configuration, there is a condition for the hooks.ortb2-blocking.enabled to be set to true in order for the module to be loaded.

This appears to not be mentioned in the existing documentation for the module.
🏷 Type of documentation

[ ] new bid adapter
[ ] update bid adapter
[ ] new feature
[ ] text edit only (wording, typos)
[x] bugfix (code examples)
[x] new examples

📋 Checklist

[ ] Related pull requests in prebid.js or server are linked -> Paste link in this list or reference it on the PR itself
[ ] For new adapters check submitting your adapter docs

Thanks @armando-fs - checking with the engineering team to confirm.


[Fix] HttpResponseMessage.Content.Header won't get header with TryGetValues

#515 introduced a regression with MobileServiceHttpClient where all the end-to-end tests on UWP are failing because this line returns false when Content-Length is not zero:
response.Content.Headers.TryGetValues("Content-Length", out contentLengthHeader)

The simpler way is to use ContentLength property of http response content header.

I didn't read #515 carefully before creating this PR. I just realize this PR is reverting the change in #515 which means it won't help folks who have this issue on newer xamarin.
Will further investigate this.


[Ballerina Shell] Shell CLI integrator tests should be enabled

Description:
Shell CLI tests were disabled by #28079  due to inconsistency with test case passing.
These tests need to be fixed and re-enabled.
Steps to reproduce:
Testing shell-cli in some computers will yield in following errors.
ballerina-shell-cli-test-suite > ballerina-shell-cli-test > io.ballerina.shell.cli.test.integration.FunctionEvaluatorTest > testEvaluatePointers STANDARD_ERROR
    Exception in thread "Thread-59" java.lang.AssertionError: Define function named add. expected [] but found [>
    ]
        at org.testng.Assert.fail(Assert.java:96)
        at org.testng.Assert.failNotEquals(Assert.java:776)
        at org.testng.Assert.assertEqualsImpl(Assert.java:137)
        at org.testng.Assert.assertEquals(Assert.java:118)
        at org.testng.Assert.assertEquals(Assert.java:453)
        at io.ballerina.shell.cli.test.base.TestIntegrator.run(TestIntegrator.java:102)

ballerina-shell-cli-test-suite > ballerina-shell-cli-test > io.ballerina.shell.cli.test.integration.FunctionEvaluatorTest > testEvaluatePointers FAILED
    java.io.IOError: java.io.IOException: Pipe broken
        at org.jline.keymap.BindingReader.readCharacter(BindingReader.java:143)
        at org.jline.keymap.BindingReader.readBinding(BindingReader.java:110)
        at org.jline.keymap.BindingReader.readBinding(BindingReader.java:61)
        at org.jline.reader.impl.LineReaderImpl.doReadBinding(LineReaderImpl.java:839)
        at org.jline.reader.impl.LineReaderImpl.readBinding(LineReaderImpl.java:859)
        at org.jline.reader.impl.LineReaderImpl.readLine(LineReaderImpl.java:586)
        at org.jline.reader.impl.LineReaderImpl.readLine(LineReaderImpl.java:417)
        at io.ballerina.shell.cli.jline.DumbJlineTerminalAdapter.readLine(DumbJlineTerminalAdapter.java:44)
        at io.ballerina.shell.cli.BallerinaShell.run(BallerinaShell.java:101)
        at io.ballerina.shell.cli.ReplShellApplication.execute(ReplShellApplication.java:88)
        at io.ballerina.shell.cli.test.integration.AbstractIntegrationTest.test(AbstractIntegrationTest.java:63)
        at io.ballerina.shell.cli.test.integration.FunctionEvaluatorTest.testEvaluatePointers(FunctionEvaluatorTest.java:45)

        Caused by:
        java.io.IOException: Pipe broken

@kdsuneraavinash can you please describe more about the statement "some computers".
According to the  #31553 It seems with the current ballerina master, shell-cli tests are working in both Windows and Ubuntu instances.
Is passing tests for both Windows and Ubuntu instances sufficient to close this issue ?

The issue causing the error was fixed in #28144. We decided to enable tests cases later, after locally testing in mac/windows, but I think it later got forgotten. So it should be safe to enable test cases now I think.

Closing since tests were enabled by #31553


this plugin influences the result of "optimize imports"

when using this plugin the result of "optimize imports" is different than expected
What steps will reproduce the issue?

Configure sort order for imports (Settings > Editor > Code Style > Java > Imports > Import Layout):
import all other imports
<blank line>
import static all other imports
enable Eclipse Code Formatter (e.g. Formatter.xml.txt)
create a file with static and non static imports, like

import java.util.ArrayList;

import static java.util.Arrays.asList;

public class Debug {

    public static void main(String[] args) {
        asList("a");
        new ArrayList<>();
    }
}

What is the expected result?
order of imports
import java.util.ArrayList;

import static java.util.Arrays.asList;

What happens instead?
order of imports
import static java.util.Arrays.asList;

import java.util.ArrayList;

system:
IntelliJ IDEA 2019.3 (Community Edition)
Build #IC-193.5233.102, built on November 28, 2019
Runtime version: 11.0.4+10-b520.11 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Windows 10 10.0
GC: ParNew, ConcurrentMarkSweep
Memory: 1952M
Cores: 12
Registry:
Non-Bundled Plugins: EclipseCodeFormatter

Yes, because that's what Eclipse does.
See File | Settings | Other Settings | Eclipse Code Formatter | Import order


Can I close the Automatically Check on :w?

Once I open a big php file, the auto check will be very slow, and i usually use :w during coding, the auto check was so boring. can i close it and call the check by :SyntasticCheck？

i have solve this problem, by
let g:syntastic_mode_map = { 'mode': 'active', 'passive_filetypes': ['php'] }