Video processing problem
Since the beginning of UNA, I have this problem. Short videos, for example 12 MB are processed, but longer videos (65 MB) remain indefinitely in process.
I looked a lot on this problem, without finding a solution. Yesterday I did the same test on the site of Amitesh Kumar and it has exactly the same problem. We are both on Ubuntu.
I said, I have a dolphin site that runs on the same server, and the same test with the same video is ok.
I did the same test here with the same video on una.io, it's ok.
An idea?
-
Hello Baloo!
The usual reasons of the similar problems - not enough resources and time to process video. SO you need to check the error log first. And show us the parameters of your PHP server - for beginning max execution time and memory limit.
With the best regards, Leonid -
Ok Leonid, I tried to increase, now memory_limit = 2048M and max_execution_time = 500 without success ... What do you advise me as value?
-
Any updates on these problems? Same thing happened to me. I was on a shared server and my host was cutting my resources back to almost nothing. So I upgraded plans. Still did not work except on the tiniest videos.
They claimed to have the ffmpeg processor on their server but I don't think they did and was not able to determine if they indeed have it available... Does anyone know how I can address the server to find out? Thanks.
Remember, you can upload videos with the Files module but it does not create multiple video formats. Also, you can do Remote Processing with the Video module if you have the connected hardware available. Other modules allow video upload, too.
Boonex offers their own hosting plans where these issues, of course, do not occur. I changed Hosts two times - both of which claimed full support of UNA. Still, no luck.
-
Well, then it's necessary to check error (and possible access) log on the upload's time. What is the minimal size of the fine working video?
-
Well, shared hosting isn't quite good for the similar engines like Una. Also - may you please provide the links on hosting plans where you had the failures with videos? And your current plan? Also you may provide me via messenger the access to your Control Panel of your hosting and the operator access to your UNA.
-
Leonid, should I install ffmpeg for php on my server, or ffmpeg.exe work alone?
Up: Also how to explain that it works on my dolphin site which is on the same server in a directory just next?
-
Fatal error: Allowed #memory size of 134217728 bytes exhausted (tried to allocate 68175096 bytes) in /var/www/una/inc/utils.inc.php on line 881
line 881 in utils.inc.php = $sResult = curl_exec($rConnect);
Conversions:
- PHP: Fatal Error: Allowed Memory Size of 8388608 Bytes Exhausted - 8 MB
- PHP: Fatal Error: Allowed Memory Size of 16777216 Bytes Exhausted - 16 MB
- PHP: Fatal Error: Allowed Memory Size of 33554432 Bytes Exhausted - 32 MB
- PHP: Fatal Error: Allowed Memory Size of 67108864 Bytes Exhausted - 64 MB
- PHP: Fatal Error: Allowed Memory Size of 134217728 Bytes Exhausted - 128 MB
- PHP: Fatal Error: Allowed Memory Size of 268435456 Bytes Exhausted - 256 MB
- PHP: Fatal Error: Allowed Memory Size of 536870912 Bytes Exhausted - 512 MB
- PHP: Fatal Error: Allowed Memory Size of 1073741824 Bytes Exhausted - 1 GB
I can set what I want as value to #memory_limit I always have the same error message!
There I am at 2048M see php.info attached. So why does he tell me in the error message that it exceeds 128 MB ??? -
Ok, finally fixed for me. I had to add ... in /inc/utils.inc.php
ini_set("memory_limit",'512M');
Now I have no more error and it processes a 130 MB MP4 in 8 minutes. I do not tell you how happy I am, it's been 6 months since I'm looking for the solution to this problem.
Amitesh Kumar try this at your house, and tell me if it works for you too.Strange problem anyway, I did not have to do that in Dolphin running on the same server, maybe Una Team can explain that?
-
Wonderful news!!! We are so glad you did not give up!
utils.inc.php is a large document, I see. Where did you add it?
Can someone official at UNA confirm this solution?
-
Hello everybody!
So as we mentioned before - the error log can help us to find what's wrong. Like Baloo found :-) So let's review the possible points of the similar situations:
1) the changes for PHP didn't take an effect, memory limit was still 128 Mb. So be sure that this setting has the necessary value. The easiest way to check it - to visit Studio->Dashboard->Host Tools->Server audit area. If you still have the wrong value - it's time to ask your hosting provider. Or you may apply the Baloo's fix but it may not work according to the hosting policy. So it would require to ask hosting support anyway.
2) why "it was working in Dolphin" and "doesn't work in UNA in the same server"
As you may check, Dolphin and UNA have the different versions of FFmpeg tool (see folders flash\modules\global\app\ for Dolphin and plugins\ffmpeg\ for UNA). UNA requires more advanced version due to more advanced work with the media. And the advanced version requires more resources. So if the first way to fix isn't available you may try to apply the one from the following set:
a) most easy, but most unreliable - just to replace FFmpeg from UNA on FFmpeg for Dolphin.
b) most complex way - take the new FFmpeg package from the https://www.ffmpeg.org/download.html site and try to change your current FFmpeg.
In both ways don't forget to backup your current FFmpeg package
-
Hello Leonid, you know, I tried Dolphin's #ffmpeg.exe, I also tried the latest version available on the official site and I tried 36,000 things without success and I even tried to pass this value via .htacces without success.
Besides, I did not really believe it when I tried this solution because it does not make sense to pass this value through the code, while memory_limit is much bigger in php.ini. that's the real mystery in the story.
So ffmpeg.exe from Una consumes more, I'm happy to learn, at least that's logical.
Suggestion:
So since I use Una, the video has never worked for me, and yet on that time, I made a server under Debian and the last on Ubuntu. ffmpeg is used by Album, Video and Messenger modules.
Other people already have or will have a problem with ffmpeg, someone has already said here that he was not supported by his host.
So I think it would be very useful to be able to disable video upload in general, you can not ask users, not to install messenger and Album if they have a problem with ffmpeg.
Having white rectangles with three black dots all over the site, is not very elegant either.
Maybe you can prevent it via "Permission". Double advantage, I would be delighted to offer the video service only to members "Premium". In addition, I could disable it for all levels in case of problems. Remember, no one is safe from a problem, even if it is only momentary.
What do you think? -
Up: It may be useful for me to give my php configuration for comparison if someone has this problem. Here it is attached.
-
Baloo, please post the relevant settings of your php config. This problem just goes on and on and on...
People want, need, and expect VIDEO. (Look at Tic Tok, for example.)
There is also in ffmpeg a "-threads" option. Has anyone adjusted it in order to lessen the big spike in resources??? Of course the transcoding will take longer but it might keep the operation from FAILING.
Thank YOU!
-
Hello banister !
So you have everything fine but still has troubles with video conversion?
-
interesting Baloo .. what define did you add this to in utils.inc.php ?
i am to understand this limits the total passengers that can get into the car at once?
-
Sliding through here to say wasssup!!! Keep it up guys!!
-
" wazzzaaaaaaaap ! " :) heh heh heh :D
-
" wazzzaaaaaaaap ! " :) heh heh heh :D
LOL, i'm out of here cause you guys are in deep mode trying to solve an issue, may the force be with you all!!!
-
Will Monte dude, Baloo 's hack worked. I think the problem is somewhat simple in the outside of the toaster-
my theory based on everything i've read, and been hacking away at the last few days is this:
somehow when cron runs for checking the conversion queue, and there is still things to be converted, it doesn't exit php right away due to the PID being locked, after passing the arguments to ffmpeg. instead it process locks the pid that runs php7x.cgi, and will not kill the worker automatically when the work queue is completed. instead it spawns a new process each minute until the conversion que has been entirely parsed, and will try to spawn as many new workers as it wants, if it loses track of the PID count that is enque and grows memory usage very rapidly under multiple concurrent upload scenario. the locks don't auto flush when worker should be done, somehow, and a check could be initiated from minute to minute, when cron runs, and there is still conversion work to be done, it can be kept track of with additional code or some architectural changes.
if this takes longer than one minute, well, every time someone on the site uploads another video, it will continue to do this behavior, eventually exhausting php's allotted memory, and gets you the dreaded email from cron telling you php has exhausted the memory. the reason it quotes default values for this, is because it has not been able to find the extra local values, or even parse php.ini at the time of crash, and so it quotes php's fallback values, of 94 megabytes, and 64 megabytes - because at this point, php has actually crashed too many concurrent threads, again memory management in the subsystem architecture of movings around..
by adding ini_set into the utils file, with a value of 512M, it essentially sandboxes una into only using 512 megs for all php thread workers that get assigned a PID and work within the context of a conversion process, and makes una wait to assign any additional work to php until it can check out each file individually that exists in the sys_transcoder_queue, which safely prevents una from crashing php by accident, because it's excited to get things done, and does not release the lock and flush the PID workers in php.cgi... i wonder if this is any different in behaviour in non fast cgi, and or if there is a setting in phpfwm that can alleviate this issue entirely...
almost positive the crash happens because una tries to launch too many php instances.
interestingly, even after the conversions are done, the PID is not released / flushed to be unlocked, because right now I have 6 versions of php73.cgi running when I check top... this is , ultimately a memory management issue that causes una to be recommended to be used with such high gig cloud servers, I feel... I may be entirely wrong here, but the memory management of the scripts could be written to avoid all of this.. and when a site scales up to be very large, it is evident as to why this is a big issue.
of course, I'm not a programmer, more of an admin type and an artist, but i see the larger patterns in things, and if the process management and memory management of the behaviours for php73 to be checked again, then flushed after the threadworker is done his job, (perhaps call an init to the register that holds the work queue from mysql db, and if 0 in mysql under sys_transcoder_queue, then flush all php7x.cgi workers, and release the memory, and release the pid... ) pardon my non programmer understanding of the correct semantic rhetoric please, it will re-spawn on demand anyway as needed based on enduser STDIN from the website.... not to worry of losing things, if this is done correctly.
cc: Anton L , LeonidS , Alex T⚜️ -
Hello Omar Amer !
Yes, it's the way to solve the lack of memory in the servers with several sites there. But it's better to have VPS or dedicated for the one if you plan to work with videos actively.
7A0CAFE1-0ECD-4EE8-9B9E-42716E421351.png